A  SURVEY  OF  OPERATIONS  RESEARCH  TECHNIQUE 
USAGE  IN  THE  U.S.   MARINE  CORPS 


Michael  C.  Mitchell 


NAVAL  POSTGRADUATE  SCHOOL 

Monterey,  California 


THESIS 


A 

SURVEY  OF 

OPERATIONS 

RESEARCH 

TECHNIQUE 

USAGE 

IN  THE 

U.S. 
by 

MARINE 

CORPS 

Michael 

C.  Mitchell 

March  1981 

Thesis  Advisor 

G .  Thomas 

Approved  for  public  release;  distribution  unlimited 


I  1  / '  / j 


ST  A 


UNCLASSIFIED 


SECURITY  CLASSIFICATION  OF  THIS  FUSE  (When  Dmo  Bnfrod) 


REPORT  DOCUMENTATION  PAGE 


1       ACPOMT    MUMttR 


READ  INSTRUCTIONS 
BEFORE  COMPLETING  FORM 


2.  OOVT   ACCESSION  NO 


1-     RECIPIENT'S  CATALOG  NUMBER 


4.     TITLE  (and  Subiitlt) 


A  Survey  of  Operations  Research 
Technique  Usage  in  the  U.S.  Marine  Corps 


5.     TYPE  OF   REPORT  A  PERIOO  COVEREO 

Master's  thesis; 
March  19  81 


••  PERFORMING  ORG.  REPORT  NUMBER 


7.  AUTHOR^*; 


Michael  C.  Mitchell 


»      CONTRACT  OR  GRANT  NLMSERfaj 


»      PERFORMING  ORGANIZATION  NAME  ANO  AOORCSS 

Naval  Postgraduate  School 
Monterey,  California   93940 


10.    PROGRAM  ELEMENT.  PROJECT     TASK 
AREA  A   WORK  UNIT  NUMBERS  ' 


I  I.     CONTROLLING  OFFICE  NAME  ANO  AOORCSS 

Naval  Postgraduate  School 
Monterey,  California   93940 


12.     REPORT  DATE 

March    1981 


II.     NUMBER  OF  PAGES 


104 


14.    MONITORING  AGENCY  NAME  A   ADDRESS^*  dlllormnt  from  Controlling  Olliea) 


tS.     SECURITY  CLASS,  (ol  Ihla  riport) 

Unclassified 


15*.     OECLASSIFI  CATION/ DOWN  GRADING 
SCHEDULE 


16.    DISTRIBUTION  STATEMENT  (ol  tntt  Xoport) 


Approved  for  public  release;  distribution  unlimited. 


17.     DISTRIBUTION  STATEMENT  (ol  tho  atmtrmet  mrtrad  In  Block  30,  II  dlllormnt  from  Hoport) 


IS.     SUPPLEMENTARY  NOTES 


IS.    KEY  WORDS  (Continue  on  rormtao  old*  II  nmtooamry  and  idonHlr  my  6JoeJr  nummor) 

Operations  Research  Technique  Usage 
Operations  Analyst  (MOS  code  9650) 
U.S.  Marine  Corps 


20.     ABSTRACT  (Contlnum  on  to*mt—  aldo  II  noeaaamvy  and  Iwmntlty  my  mtmck  numoar) 

This  thesis  presents  the  results  of  a  mailed  survey  of 
all  officers  currently  serving  on  active  duty  in  the  United 
States  Marine  Corps  who  possess  a  secondary  Military 
Occupational  Specialty  (MOS)  of  Operations  Analyst  (MOS  code 
9650) .   The  questionnaire  used  was  designed  to  determine  the 
usage  frequency  and  the  relative  importance  of  each  of 
35  Operations  Research  (OR)  techniques  and  application  areas 


do  ,: 


FORM 
AN  73 


1473  EDITION  OP  1  MOV  «•  IS  OBSOLETE 

S/N    0  10  2-014-6601    | 


UNCLASSIFIED 


IBCURITY  CLASSIFICATION  OF  TNIS  PAOt  (Whon  Dot*  Kntotod) 


UNCLASSIFIED 


,  wtv  CL.  AMI>tC*TlOjj  Qg  Twit  »*««<■■%—  jjggg  «■——«• 


(20.   ABSTRACT   Continued) 


to  these  Marine  officers  during  their  most  recent  tours  in 
9650  billets.   In  addition ,  the  results  were  classified 
according  to  individual  service  rank,  function  area  and 
type  of  OR  work  performed  and  a  series  of  nonparametric 
statistical  tests  was  conducted  to  determine  significant 
differences.   Comments  elicited  by  the  survey  were  also 
analyzed  and  recommendations  were  made  based  upon  the 
conclusions  drawn. 


Form   1473 
1  Jan  73 
N  0102-014-6601 


UNCLASSIFIED 


tteuftiw  cuamiucatioh  a*  ▼*•»  •*•£*■»•<•  ©•*•  §»••*•** 


Approved  for  public  release;  distribution  unlimited 


A  Survey  of  Operations  Research  Technique 
Usage  in  the  U.S.  Marine  Corps 

by 

Michael  C.  Mitchell 

Captain,  United  States  Marine  Corps 

B.S.,  United  States  Naval  Academy,  1973 


Submitted  in  partial  fulfillment  of  the 
requirements  for  the  degree  of 


MASTER  OF  SCIENCE  IN  OPERATIONS  RESEARCH 

from  the 

NAVAL  POSTGRADUATE  SCHOOL 
March  1981 


HV 


ABSTRACT 

This  thesis  presents  the  results  of  a  mailed  survey  of 
all  officers  currently  serving  on  active  duty  in  the  United 
States  Marine  Corps  who  possess  a  secondary  Military  Occupa- 
tional Specialty  (MOS)  of  Operations  Analyst  (MOS  code  9650) . 
The  questionnaire  used  was  designed  to  determine  the  usage 
frequency  and  the  relative  importance  of  each  of  35  Operations 
Research  (OR)  techniques  and  application  areas  to  these  Marine 
officers  during  their  most  recent  tours  in  9650  billets.   In 
addition,  the  results  were  classified  according  to  individual 
service  rank,  function  area  and  type  of  OR  work  performed  and 
a  series  of  nonpar ame trie  statistical  tests  was  conducted  to 
determine  significant  differences.   Comments  elicited  by  the 
survey  were  also  analyzed  and  recommendations  were  made  based 
upon  the  conclusions  drawn. 
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I.   BACKGROUND 

A.  INTRODUCTION 

This  thesis  presents  the  results  of  a  mailed  survey  of 
all  officers  currently  serving  on  active  duty  in  the  United 
States  Marine  Corps  who  possess  a  secondary  Military  Occupa- 
tional Specialty  (MOS)  of  Operations  Analyst  (MOS  code  9650) . 
The  questionnaire  used  was  designed  to  determine  the  usage 
frequency  and  the  relative  importance  of  each  of  35  Operations 
Research  (OR)  techniques  and  application  areas  to  these  Marine 
officers  during  their  most  recent  tours  in  9650  billets.   In 
addition,  the  results  were  classified  according  to  individual 
service  rank,  function  area  and  type  of  OR  work  performed  and 
a  series  of  statistical  tests  was  conducted  to  determine 
significant  differences.   Comments  elicited  by  the  survey 
were  also  analyzed  and  recommendations  were  made  based  upon 
the  conclusions  drawn. 

B.  HISTORY  OF  OPERATIONS  ANALYSIS  IN  THE  MARINE  CORPS 

The  Marine  Corps  initiated  an  "in-house"  operations  analy- 
sis capability  in  1957,  when  a  civil  service  billet  was  estab- 
lished at  Marine  Corps  Headquarters  (HQMC)  in  Washington, 
D.C.  [Ref .  1] .   Subsequently,  the  Commandant  of  the  Marine 
Corps  decided  that  there  existed  a  need  for  Marines  educated 
at  the  graduate  level  in  the  field  of  Operations  Research/ 
Systems  Analysis  (OR/SA)  so,  in  1958,  the  first  Marine  officer 
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was  assigned  to  the  Naval  Postgraduate  School  (NPS)  to  study 
their  Operations  Research  curriculum.   During  the  next  few 
years  only  about  four  or  five  Operations  Analyst  billets  were 
identified  by  the  Marine  Corps,  consequently,  the  input  of 
Marines  to  the  NPS  program  was  small,  averaging  only  about 
two  per  annum  through  1963.   Although  limited  operations  analy- 
sis support  was  available  to  the  Marine  Corps  through  the 
Navy's  CNO  Operations  Evaluation  Group,  the  Marines  did  not 
possess  a  significant  analysis  capability  until  1965,  when 
the  Marine  Corps  Operations  Analysis  Group  was  established  as 
a  separate  division  at  the  Center  for  Naval  Analysis,  a  pri- 
vate, non-profit  organization  under  contract  to  the  Navy  and 
administered   by  the  University  of  Rochester.   That  same  year, 
the  first  Fleet  Marine  Force  (FMF)  operations  analysis  sec- 
tion was  formed  at  FMF  Pacific. 

During  the  middle  sixties  the  requirements  for  Marine 
Operations  Analysts  accelerated  rapidly  due  to  an  increased 
recognition  of  the  value  of  analytical  problem  solving. 
Correspondingly,  the  annual  personnel  input  to  the  OR/SA 
curriculum  at  postgraduate  school  increased  until  the  current 
level  of  about  10  officers  per  year  was  reached.   Although  the 
great  majority  of  these  officers  attend  the  Naval  Postgraduate 
School  at  Monterey,  California,  a  few  also  attend  civilian 
universities.   Presently,  there  are  19  Marine  officers  enrolled 
in  the  OR/SA  curriculum  at  the  Naval  Postgraduate  School  and 
2  enrolled  at  civilian  schools. 
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These  personnel,  once  educated,  will  be  assigned  the 

Military  Occupational  Specialty  code  9  650  and  given  the  title 

Operations  Analyst.   The  Marine  Corps  MOS  Manual  describes 

the  duties  and  tasks  which  these  people  should  be  capable  of 

completing  as  follows: 

Duties  and  Tasks;   Participates  in  the  following: 
construction  of  models,  manual  or  computer,  for 
analysis  or  comparison  of  military  activities, 
operations,  weapons  systems,  and  force  structures; 
the  gathering  of  empirical  data  used  in  support  of 
analysis,  comparison,  or  war  gaming  of  military 
activities,  weapons  systems  and  plans;  cost  effective- 
ness studies  to  analyze  weapons  systems,  operational 
concepts,  and  resource  allocation.   Conducts  sta- 
tistical evaluations  and  comparisons  of  the  results 
of  such  tests  and  writes  technical  reports  on  results. 
Directs,  supervises,  conducts,  or  participates  in  war 
gaming  or  simulation  activities,  including  preparation 
of  rules  or  flow  charts,  conduct  of  game  or  simulation, 
assessment  of  the  game  moves,  of  simulation  steps, 
evaluation  of  the  results,  and  determination  of  appro- 
priate measures  of  effectiveness.   Performs  technical 
liaison  between  military  organizations  and  activities 
engaged  in  analytical  or  evaluation  work  and  techni- 
cal assistance  [for]  contractors  supporting  the 
military  activity.   Performs  any  or  all  of  the  above 
duties  and  tasks  at  research  and  development  activi-. 
ties,  management  activities,  or  with  operational 
units  of  the  Fleet  Marine  Force.   [Ref.  2] 

In  less  sophisticated  terms,  a  Marine  Operations  Analyst 

is  an  individual  capable  of  performing  operations  analysis, 

where  such  analysis,  as  defined  by  the  Marine  Corps,  is  "the 

application  of  scientific  methods  to  perform  analysis  directed 

toward  improving  the  operational  effectiveness  of  organizations 

and  the  operating  efficiency  of  organized  man-machine  systems 

in  achieving  their  objectives"  [Ref.  3:p.  1] .   While  the  term 

"operations  analysis"  and  "operations  research"  are  synonymous, 

the  term  "systems  analysis"  carries  a  slightly  different 
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connotation,  being  "the  application  of  scientific  methods  to 
assist  a  decision  maker  to  choose  among  alternatives  by  a 
systematic  investigation  of  the  relevant  costs,  risk  and  meaning- 
ful effectiveness  of  alternative  systems,  equipments  or 
organizations"  [Ref.  3:p.  2]. 

Operations  analysis  is  used  to  address  technical,  tactical 
problems  while  systems  analysis  often  addresses  the  more 
loosely  defined  strategic  problems.   The  essence  of  an  Opera- 
tions Analyst's  work  is  to  provide  a  decision  maker  with  an 
analytical  method  for  choosing  among  alternative  procedures, 
allocations  and/or  courses  of  action. 

We  can  gain  a  better  idea  of  operations  analysis  in  the 
Marine  Corps  by  reviewing  how  analysts  are  trained. 

C.   MARINE  CORPS  SPECIAL  EDUCATION 

To  support  the  educational  requirement  for  trained  Opera- 
tions Analysts,  the  Marine  Corps  has  two  advanced  education 
programs:   the  Special  Education  Program  (SEP)  [Ref.  4]  and 
the  Advanced  Degree  Program  (ADP)  [Ref.  5]. 

The  Special  Education  Program  allows  voluntary  selectees 
to  attend  either  the  Naval  Postgraduate  School  or  one  of  a 
limited  number  of  specified  civilian  universities  in  order 
to  study  for  an  advanced  degree  in  a  number  of  selected  fields, 
including  OR/SA.   The  Marine  Corps  pays  all  tuition  and  re- 
quired academic  fees  as  well  as  providing  normal  pay  and 
allowances  while  the  officer  is  attending  school. 

The  Advanced  Degree  Program  was  established  to  augment 
the  SEP  by  providing  an  additional  means  for  career  officers 
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to  obtain  a  graduate  education  in  order  to  qualify  for  SEP 
billets.   Through  this  program,  officers  may  voluntarily 
apply  to  civilian  universities  acceptable  to  the  Marine  Corps 
to  study  in  a  field  in  which  the  Corps  has  realized  a  per- 
sonnel shortage.   During  the  course  of  attendance,  each  offi- 
cer continues  to  receive  his  or  her  normal  pay  and  allowances 
from  the  Marine  Corps  but  must  bear  the  cost  of  tuition  and 
textbooks . 

Officers  applying  for  either  program  must  agree  not  to 
submit  a  resignation  nor  to  request  separation  while  partici- 
pating in  the  program  and  must  further  agree  to  remain  on  active 
duty,  after  completion  of  their  education  or  upon  separation 
from  the  program,  for  a  period  of  3  to  4  years  (depending  upon 
the  length  of  their  course  of  instruction).   Upon  successful 
completion  of  one  of  these  programs,  the  officers  are  assigned 
a  secondary  MOS  and  are  sent  to  SEP  billets  requiring  the  use 
of  their  newly  acquired  skills.   (The  only  exceptions  to  this 
immediate  SEP  assignment  policy  are  officers  who  are  scheduled 
for  an  overseas  assignment  or  officers  who  have  studied  in 
a  field  in  which  no  billets  are  currently  available) .   In  any 
event,  an  officer  with  a  SEP  MOS  can  expect  to  be  required  to 
serve  alternating  tours  in  SEP  billets  thereafter. 

D.   SPECIAL  EDUCATION  PROGRAM  BILLETS 

The  Marine  Corps  currently  (1980)  has  414  billets  desig- 
nated as  Special  Education  Program  (SEP)  billets,  the  number 
of  billets  authorized  at  any  time  being  the  result  of  an 
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iterative  justification  process.   Recommendations  for  estab- 
lishment of  SEP  billets  can  be  made  to  the  Deputy  Chief  of 
Staff  for  Manpower  by 

1)  Marine  Corps  field  commanders  (Divisions,  Wings,  etc.) 

2)  SEP  discipline  sponsors 

3)  Table  of  Organization  (T/0)  sponsors 

SEP  discipline  sponsors  are  the  HQMC  staff  sections  desig- 
nated as  the  "in-house"  expert  on  their  particular  discipline. 
(The  SEP  discipline  sponsor  for  OR/SA  is  the  Deputy  Chief  of 
Staff  for  Research,  Development  and  Studies.)   The  T/0  sponsor 
is  the  HQMC  staff  section  with  administrative  responsibility 
for  the  Table  of  Organization  which  includes  the  recommended 
billet. 

Specific  requirements  and  justification  for  the  establish- 
ment of  a  new  billet  must  be  submitted  in  the  form  of  a  com- 
pleted Billet  Education  Evaluation  Certificate.   In  order  for 
a  billet  to  be  identified  as  requiring  graduate  level  educa- 
tion, it  must  be  determined  that  such  a  level  is  either 
"Necessary"  or  "Desirable"  for  the  proper  functioning  of  the 
billet  incumbent.   The  classification  of  "Necessary"  is  assigned 
only  to  billets  meeting  one  of  the  following  criteria:   [Ref.  6: 
p.  2] 

1)  Billets  which  are  required  by  law  or  DOD  policy  to  be 
filled  by  individuals  possessing  graduate  level 
education  in  a  specific  field  of  study. 

2)  Billets  in  which  the  primary  duties  of  the  incumbent 
cannot  be  satisfactorily  performed  except  by 
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individuals  possessing  qualifications  which  can  be 
acquired  only  through  graduate  education  in  a  relevant 
field  of  study. 
3)   Billets  which  must  be  filled  by  individuals  who  are 
required  to  exert  direct  technical  supervision  over 
military  or  civilian  billets  with  graduate  education 
prerequisites . 
The  classification  of  "Desirable"  is  assigned  to  billets  not 
meeting  any  of  the  above  criteria,  but  which  can  be  most 
effectively  filled  by  individuals  who  have  had  graduate  training 

In  the  billet  approval  process  the  discipline  sponsor  has 
the  major  responsibility  for  determining  if  recommended  billets 
actually  require  graduate  education.   In  addition,  the  sponsor 
is  required  to  conduct  an  annual  review  of  all  current  billet 
educational  requirements  to  ensure  that  each  educational  level 
is  continually  warrented. 

There  are,  presently,  53  SEP  billets  specified  as  being 
Operations  Analyst  billets.   (The  number  of  Marine  Operations 
Analysts  currently  on  active  duty  is  77,  well  below  the  127 
necessary  in  order  to  meet  the  Marine  Corps  desired  level  of 
2.4  trained  officers  for  each  billet.)   The  Marine  Corps 
Operations  Analysis  Group  located  at  the  Center  for  Naval 
Analysis  currently  represents  the  largest  single  OR/SA  capa- 
bility in  the  Marine  Corps.   This  organization,  comprised 
of  civilian  and  Marine  Operations  Analysts  and  support  per- 
sonnel, provides  a  detachment  of  analysts  to  the  Marine  Corps 
Development  and  Education  Center  (MCDEC)  and  one  analyst  to 
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each  of  the  FMF  Headquarters.  Additionally,  smaller  opera- 
tions analysis  sections  consisting  of  military  and/or  civil 
service  Operations  Analysts  have  been  established  as  well  as 
certain  individual  billets.  The  majority  of  these  sections 
and  billets  are  currently  located  at  HQMC  and  MCDEC  although 
there  are  also  individuals  located  with  joint  commands  such 
as  the  Department  of  Defense,  the  Joint  Chiefs  of  Staff  and 
the  Defense  Communications  Agency. 

The  majority  of  Operations  Analyst  billets  being  spread 
throughout  the  Marine  Corps  rather  than  being  pooled  at  a 
single  location  reflects  the  current  Marine  Corps  policy  that 
Operations  Analysts  should  provide  direct  support  to  individual 
commanders  and  staff  section  heads.   In  fact,  "to  employ 
analysts  in  a  substructure  below  this  level  tends  to  reduce 
their  effectiveness"  [Ref.  3:p.  5], 

E.   PROBLEM  STATEMENT 

Upon  completion  of  interviews  with  several  persons  pre- 
sently assigned  to  MOS  9650  SEP  billets,  the  author  concluded 
that  the  official  Marine  Corps  explanation  of  the  tasks  and 
duties  assigned  to  these  personnel  does  not  always  reflect 
the  actual  functions  of  their  billets.   In  addition,  discussions 
with  students  enrolled  in  the  OR/SA  curriculum  at  NPS  revealed 
an  unclear  perception  of  the  real  responsibilities  and  func- 
tions of  Marine  Operations  Analysts  "in  the  field" .   This  re- 
sults from  the  lack  of  a  published,  clear  definition  of  the 
actual  roles  played  by  Marine  Operations  Analysts  and  an 


17 


explanation  of  the  techniques  that  they  employ.   What  tasks 
are  Marine  Operations  Analysts  actually  being  required  to 
complete?   Are  they  being  taught  the  techniques  that  they  will 
be  obliged  to  understand  in  order  to  complete  their  tasks? 
A  method  of  answering  these  questions  is  to  ask  the  officers 
currently  assigned  the  965  0  secondary  MOS  what  tasks  they 
have  been  required  to  complete  and  what  techniques  they  used. 
The  answers  from  such  a  survey  can  then  be  utilized  in  the 
evaluation  and  possible  modification  of  the  curriculum  em- 
ployed in  educating  potential  Marine  Operations  Analysts  at 
the  Naval  Postgraduate  School.   Additionally,  a  survey  of  OR/ 
SA  techniques  currently  in  use  by  Marine  analysts  may  provide 
a  basis  for  developing  a  program  of  elective  courses  that  might 
be  recommended  to  future  Marine  Corps  students  at  NPS. 

The  purpose  of  this  thesis  was  to  conduct  a  survey  of  the 
Operations  Research  techniques  currently  being  used  by  Marine 
9650 !s  and,  through  statistical  inference  and  comparative 
analysis,  make  recommendations  for  improving  the  training 
and  utilization  of  Marine  Operations  Analysts. 

F.   PREVIOUS  STUDIES 

Studies  of  OR  technique  usage  during  the  last  ten  years 
include  surveys  of  both  corporate  and  military  usage.   In 
1977,  Ledbetter  and  Cox  surveyed  the  500  largest  U.S.  indus- 
trial firms  as  listed  by  Fortune  magazine  in  1975  [Ref .  7]. 
Analyzing  their  176  returns  with  respect  to  the  use  of  OR 
techniques  and  the  specific  areas  of  OR  application,  they 
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found   that  a  great  majority  of  the  responding  firms  used 
regression  analysis  to  at  least  some  degree,  while  a  nearly 
like  number  used  simulation  (in  production)  and  linear  pro- 
gramming (see  Table  I) .   Simulation  and  linear  programming 
reflected  the  greatest  breadth  of  usage  over  a  wide  range  of 
application  areas  while  regression  analysis  showed  the  highest 
ranking  for  across-the-board  degree  of  usage  within  the 
firms. 

TABLE  I 
CORPORATE  OR  TECHNIQUE  USAGE  (LEDBETTER  AND  COX) 


Technique 

Regression  Analysis 
Simulation 
Linear  Programming 
Queueing  Theory 
Network  Models 
Dynamic  Programming 
Game  Theory 


%  of  Firms  Reporting  Usage 
90.5 
88.6 
84.6 
63.4 
60.9 
46.4 
40.3 


In  their  study  of  137  Case  Western  Reserve  University 
alumni,  Rasmussen  and  George  asked  the  respondents  to  report 
the  value  to  them  during  their  careers  of  various  theoretical 
and  methodological  subject  areas  related  to  the  Operations 
Research  field  [Ref.  8],   The  following  order  of  value  was 
reported  by  respondents  possessing  a  Master  of  Science  degree 
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Statistical  Methods 
Systems  Analysis 
Forecasting 
Information  Systems 
Simulation 
Decision  Theory 
Linear  Programming 
Scheduling  Theory 
Production  Management 
Inventory  Theory 
Stochastic  Processes 
Non-linear  Programming 
Integer  Programming 
Dynamic  Programming 
Queueing  Theory 

Other  techniques  and  related  subject  areas  considered  to 
be  important  by  those  answering  the  survey  included:   Networks, 
Graphs,  Reliability  Theory,  Modeling,  Case  Studies,  Problem 
Formulation,  Computing,  Cost/Benefit  Analysis,  Structure  of 
Organizations,  Philosophy  of  Management,  Communications  and 
"the  organization  survival  of  the  Operations  Research  prac- 
titioner".  Also  reflected  in  this  study  were  the  inferences 
that  Ph.D.  respondents  felt  that  the  Mathematical  Programming 
areas  were  more  useful  than  the  M.S.  respondents  did  and  that 
Ph.D.  respondents  reported  a  higher  level  of  receptivity  to 
OR  techniques  by  their  employers  than  did  the  M.S.  respondents. 

In  research  conducted  in  1977,  Thomas  and  DaCosta  [Ref.  9] 
compared  their  survey  of  150  California  firms  and  institutions 
and  "Fortune  500"  corporations  with  surveys  that  had  been  con- 
ducted by  Hovey  and  Wagner  [Ref.  10]  in  1958,  Schumacher  and 
Smith  [Ref.  11]  in  1964,  and  Turban  [Ref.  12]  in  1969.   Their 
findings  on  usage  of  OR  techniques,  as  shown  in  Table  II,  were 
similar  to  those  of  Ledbetter  and  Cox.   In  addition,  they 
found  that,  when  the  users  were  queried  as  to  the  types  of 
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TABLE  II 
CORPORATE  OR  TECHNIQUE  USAGE  (THOMAS  AND  DACOSTA) 

Technique  %  of  Firms  Reporting  Usage 

Statistical  Analysis  93 

Simulation  84 

Linear  Programming  79 

PERT/CPM  70 

Inventory  Theory  5  7 

Queueing  Theory  45 

Non-linear  Programming  36 

Heuristic  Programming  34 

Bayesian  Decision  Analysis  32 

Dynamic  Programming  27 

Risk  Analysis  3 

Integer  and  Mixed  Programming  2 

Delphi  1 

Financial  Methods  1 


difficulties  that  are  encountered  in  attempting  to  implement 
OR  work,  the  most  frequently  mentioned  difficulties  were 
user  commitment,  user  understanding  and  organizational  resistance 
to  change.   Specifically,  they  felt  that  "if  the  manager  who 
must  use  the  proffered  solution  does  not  understand  its  basis 
there  will  be  friction  and  difficulties"  [Ref.  9:p.  108]. 

All  of  the  above-noted  studies  were  related  to  private 
industry  which  has  more  easily-measured  overall  objectives 
such  as  profit  and  profit  contribution.   This  might  well  cause 
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corporate  survey  results  to  be  far  different  from  those  that 
would  be  obtained  from  an  inquiry  into  the  usage  of  OR  tech- 
niques by  military  personnel. 

In  1978,  Obert  conducted  a  telephone  survey  of  ninety 
percent  (328)  of  the  Operations  Research  and  Systems  Analysis 
positions  within  the  Army  [Ref.  13].   His  figures  for  fre- 
quency of  OR  technique  usage  for  individuals  characterized  as 
holding  OR/SA  engineer  billets  are  shown  in  Table  III.   One 
should  bear  in  mind  that  the  principal  difference  between  this 
survey  and  most  of  the  ones  previously  mentioned  is  that  in 
this  survey  individuals  were  asked  to  report  frequency  of 
usage  of  OR  techniques  while,  in  the  corporate  surveys, 
organizations  were  asked  to  report  usage  frequencies.   Obert 
found  that  almost  ninety  percent  of  his  respondents  character- 
ized their  duties  as  either  doing  original  OR/SA  research, 
correlating  the  work  of  other  researchers  or  reviewing  OR/SA 
work.   Additionally,  he  found  that  while  over  eighty  percent  of 
his  respondents  worked  with  established  computer  programs,  the 
majority  did  not  do  their  own  computer  programming  (FORTRAN 
being  the  programming  language  most  frequently  employed  by 
the  program-writing  minority) .   Finally,  it  was  determined 
that,  while  the  overall  results  for  OR/SA  engineers  showed  that 
almost  three-fourths  of  the  respondents  said  that  they  spent 
one-half  or  less  of  their  time  using  quantitative  analysis 
or  OR/SA  techniques  compared  to  their  other  daily  duties, 
Captains  and  Majors,  apparently,  spent  more  of  their  time 
doing  analysis  than  did  Lieutenant  Colonels  and  Colonels. 
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TABLE  III 
OR  TECHNIQUE  USAGE  BY  ARMY  QR/SA  ENGINEERS 

Technique  %  of  Individuals  Reporting  Usage 

Parametric  Statistics  98.5 

Probability  9  6.5 

Formal  Data  Analysis  93.9 

Measures  of  Effectiveness  78.8 

Simulation  Models  and  Applications  7  3.8 

Networks,  Flows  and  Graphs  65.6 

Combat  Models  64.6 

Decision  and  Risk  Analysis  64.2 

War  Games  63.1 

Experimental  Design  54.1 

Nonparametric  Statistics  48.5 

Human  Factors  45.4 

Stochastic  Models  44.9 

Learning  Curves  44.5 

Cost  Estimating  41.9 

Linear  Algebra  41.4 

Linear  Programming  37.9 

Calculus  32.8 

Economics  and  Economic  Analysis  25.7 

Manpower  Models  24.6 

Accounting  and  Financial  Applications  23.2 

Dynamic  and  Non-linear  Programming  21.7 

Cost  Estimating  Relationships  19.2 

Economic  Input-Output  Models  8.1 

23 


The  following  chapters  explain  the  planning,  conduct  and 
results  of  the  work  undertaken  by  the  author  in  an  attempt 
to  ascertain  the  frequency  of  usage  of  OR  techniques  in  the 
Marine  Corps.   Chapter  II  describes  the  experimental  methodology 
developed  to  obtain  measurable  results  from  a  survey  of  Marine 
Operations  Analysts.   These  results  are  presented  and  analyzed 
in  Chapter  III.   Numerical  data  obtained  from  the  question- 
naires are  categorized  into  several  groups  and  the  statisti- 
cal tests  that  were  conducted  are  described.   Comments  made 
by  the  respondents  are  grouped  according  to  theme  and  sub- 
jectively discussed.   Finally,  in  Chapter  V,   conclusions 
are  drawn  from  the  results  of  the  analysis  and  recommendations 
for  the  improved  utilization  of  9650' s  are  made. 
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II.   METHODOLOGY  AND  CONDUCT  OF  THE  EXPERIMENT 

Since  the  experiment  population  was  well  defined  (all 
Marine  personnel  holding  a  MOS  of  9650  and  currently  on  active 
duty)  and  equally  easy  to  locate,  it  was  decided  that  a  mailed 
questionnaire,  as  opposed  to  a  telephone  survey,  would  be  the 
most  appropriate  method  of  conducting  the  survey.   This  would 
provide  a  controlled,  unbiased  means  of  obtaining  answers 
while  allowing  individuals  the  time  necessary  to  think  about 
each  question  before  responding.   Since  a  mailed  questionnaire 
might  result  in  a  lower  response  rate  than  could  be  obtained 
via  personal  contact,  it  was  decided  that,  upon  reaching  a 
return  rate  of  75%,  the  remaining  individuals  who  had  not 
responded  would  be  contacted  by  telephone  to  determine  if  they 
required  additional  information  or  assistance.   A  final  response 
rate  goal  of  90%  was  deemed  to  be  reasonable  since  it  was  felt 
that  at  least  10%  of  the  population  might  not  be  able  to  be 
reached  due  to  being  deployed  overseas,  enroute  to  a  new  duty 
station,  on  leave,  etc. 

The  questionnaire  was  designed  so  that  each  question  would 
obtain  replies  which  could  be  summarized  in  a  fashion  capable 
of  answering  questions  implied  by  the  thesis  objectives  (see 
Appendix  A  for  questionnaire  format) .   The  first  four  questions 
were  directly  related  to  the  respondents'  billets,  requesting 
the  billet  title,  location  and  general  function  area  as  well 
as  the  service  rank  of  the  individual  and  the  type  of  OR  work 
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that  the  billet  required  him  to  do  the  most.   The  answers 
obtained  from  these  questions  would  be  used  to  group  the 
returns  for  detailed  study  during  their  analysis.   Questions 
5  and  6  were  related  to  computer  programming  and  the  use  of 
preprogrammed  computer  packages,  inquiring  as  to  the  types 
of  computer  languages  used,  if  any,  and  the  reasons  for  their 
usage.   In  an  attempt  to  secure  some  measure  of  the  usage 
frequency  and  importance  of  OR  techniques  to  the  practicing 
analysts,  questions  7  and  8  were  developed,  allowing  the 
respondents  to  rate  each  subject  according  to  their  personal 
experiences.   Question  9  was  added  as  an  attempt  to  include 
in  the  analysis  any  subject  (not  included  in  questions  7  and 
8)  that  the  respondents  used  frequently  or  felt  to  be  impor- 
tant to  them.   Question  10  was  an  attempt  to  ascertain  how 
much  of  their  work  time  analysts  are  spending  on  OR  labors. 
Question  11  would  help  to  determine  the  acceptance  level  of 
OR  work  "in  the  real  world",  while  responses  obtained  from 
question  12  would  result  in  subjective  recommendations  for 
the  better  utilization  of  Marine  Operations  Analysts. 

Recognizing  that  the  answers  obtained  would  be  clearly 
of  a  subjective  nature  and  that  the  form  of  the  underlying 
distribution  of  the  answers  would  be  unknown,  it  was  decided 
that  nonparametric  (i.e.,  distribution-free)  methods  of  statis- 
tical analysis  would  be  appropriate  for  analyzing  the  returns. 
Parametric  methods  require  that  more  assumptions  be  made  than 
do  nonparametric  methods,  consequently  nonparametric  statistical 
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tests  are  generally  less  powerful  than  parametric  statistical 
tests. 

This  power-deficiency  can  be  corrected  when  using  non- 
parametric  methods,  however,  by  drawing  a  larger  sample  size 
from  the  population  [Ref.  14  :p.  21]  (hence,  the  imperative  of 
the  return  rate  goal  of  90%) .   One  assumption  that  must  be 
made,  though,  whether  using  parametric  or  nonparametric 
methods  is  that  the  observations  are  independent.   Additionally, 
nonparametric  methods  as  used  in  this  study  require  that  all 
questions  intended  to  be  analyzed  be  designed  so  that  answers 
are  at  least  based  on  an  ordinal  scale  and  that  all  observa- 
tions are  drawn  from  an  underlying  continuous  distribution. 

The  strategy  for  the  data  analysis  was  first  to  determine 
the  overall  frequency  of  usage  and  perceived  importance  of  a 
number  of  OR  techniques  and  application  areas  and  then  to 
determine  if  there  were  differences  according  to  frequency 
of  usage  or  perceived  importance.   One  might  expect  that  there 
would  be  differences  and  this,  in  fact,  was  borne  out  by  the 
results.   A  more  difficult  question  to  answer,  however,  is 
what  are  the  possible  causes  of  the  differences.   The  data 
were  grouped  according  to  the  service  rank  of  the  individuals, 
the  functional  area  related  to  their  billets  and  the  type  of 
OR  work  that  their  billets  required  them  to  do  the  most  and 
statistical  tests  were  conducted  to  determine  possible  causes 
for  differences  and  relationships  between  the  groups.   These 
tests,  the  conclusions  drawn  from  them  and  various  recommenda- 
tions are  presented  in  the  chapters  that  follow. 
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III.   DATA  ANALYSIS 

A  copy  of  the  survey  questionnaire  contained  in  Appendix 
A  was  sent  to  each  of  the  77  Marine  officers  designated  as 
Operations  Analysts  (Military  Occupational  Specialty  9  650)  who 
are  currently  on  active  duty.   Individuals  were  asked  to  base 
all  of  their  answers  on  the  most  recent  MOS  9  650  billet  in 
which  they  had  served.   Those  who  had  never  served  in  a  MOS 
9650  billet  were  asked  to  state  so  and  return  the  unanswered 
questionnaire.   A  total  of  7  2  persons  returned  their  question- 
naires for  a  return  rate  of  93.5%.   Six  of  these  people  stated 
that  they  had  never  held  a  9650  billet,  leaving  a  usable  sam- 
ple of  66  respondents  for  analysis  (i.e.,  a  83.3%  usable  re- 
turn rate) . 

A.   RANK 

Table  IV  contains  f requency-of-answer  data  relating  indi- 
vidual service  rank.   The  reader  is  reminded  that  the  tabulations 
do  not  reflect  the  current  status  of  the  population,  but, 

TABLE  IV 
SERVICE  RANK  DURING  MOST  RECENT  9650  BILLET 
Rank  Raw  Score    (Percentage) 

Captain 
Major 

Lieutenant  Colonel 
Colonel 
Total 
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17 

(25.8) 

35  \ 

(53.0) 

12 

(18.2) 

_1 

(    3.0) 

66 

(100.0) 

rather,  the  status  of  each  individual  during  his  most  recent 
tour  in  a  9650  billet.  As  an  illustration:  40  of  the  77 
officers  sent  questionnaires  currently  hold  the  rank  of  Lieu- 
tenant Colonel  but  only  12  officers  of  the  66  respondents  re- 
ported that  they  held  that  rank  during  their  most  recent  9  650 
billet  assignment.  As  can  be  seen,  a  majority  of  the  respon- 
dents said  that  they  had  held  their  billets  as  Majors. 

B.   FUNCTION  AREA 

Respondents  were  fairly  evenly  spread  among  functional 
areas  of  employment  (see  Table  V) .   Eight  individuals  reported 
that  their  assignments  were  related  to  a  combination  of  the 
listed  function  areas  while  an  equal  number  said  that  their 
jobs  were  associated  with  a  different  field  of  work  than  those 


TABLE  V 
FUNCTIONAL  AREA 


Functional  Area 


Manpower 

Logistics 

Finance 

Operations  and  Plans 

Test  and  Evaluation 

Aviation 

Instruction 

Combination 

Other 

Total 


Raw  Score 

(Percentage) 

11 

(16.7) 

8 

(12.1) 

1 

(1.5) 

7 

(10.6) 

14 

(21.2) 

6 

(9.1) 

3 

(4.5) 

8 

(12.1) 

_8_ 

(12.1) 

66 

(100.0) 
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listed  (e.g.,  system  development,  research  and  development 
[5] ,  studies  and  long-range  planning) . 

C.   TYPE  OF  OR  WORK  DONE 

As  Table  VI  indicates,  one-third  specified  that  they  did 
original  OR  work  most  of  the  time.  A  nearly  like  number 
(34.8%)  reported  that  most  of  their  OR  work  was  done  review- 
ing work  completed  outside  of  their  sections.  Two  questions 
of  interest:  Does  the  most  frequent  type  of  OR  work  done 
vary  by  rank  and  does  the  most  frequent  type  of  OR  work  done 
vary  by  function  area? 


TABLE 

VI 

TYPE 

OF 

OR 

WORK 

DONE 

Type  of  Work 

Raw  Score 

(P< 

ercentage 

Original 

22 

(33, 

.3) 

Correlate 

4 

(6. 

,1) 

Review 

23 

(34. 

.8) 

Instructor 

3 

(4. 

.5) 

Combination 

6 

(9. 

.1) 

Other 

3 

(4. 

.5) 

None 

_5 

(7. 

.6) 

Total 

66 

(100. 

,0) 

D.   TYPE  OF  WORK  DONE  BY  RANK  OF  OFFICER 

To  answer  the  first  question,  the  null  hypothesis  that 
there  is  no  difference  in  the  frequency  distribution  of  type 
of  OR  work  done  between  the  more  junior  ranks  of  Captain  and 
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Major  and  the  more  senior  ranks  of  Lieutenant  Colonel  and 

Colonel  was  tested.   The  associated  frequency  distributions 

2 

are  shown  in  Table  VII.   A  Chi-square  (x  )  test 

indicates  that  there  is  not  a  significant  difference  between 


TABLE  VII 
SERVICE  RANK  VERSUS  TYPE  OF  OR  WORK  DONE 


Lieutenant  Colonels/    Total 
Colonels 

2  22 

0  4 
8  23 

1  3 
0  6 

2  3 
_1  _5_ 
14  66 


P  Value  =  0.084 

junior  and  senior  officers  based  upon  the  type  of  OR  work  per- 
formed (P  =  0.084).   (The  P  value  is  the  smallest  value  of 
type  I  error  probability  for  which  there  would  be  a  rejection  of  the 
null  hypothesis.   It  was  decided  by  the  author  that  a  critical 
level  of  0.05  would  be  utilized  for  the  rejection  of  any  null 
hypothesis  in  this  study) . 

It  should  be  noted  that,  in  only  two  of  the  categories 
listed,  the  expected  frequencies  are  greater  than  five.   Some 
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Captains/ 
Majors 

Original 

20 

Correlate 

4 

Review 

15 

Instructor 

2 

Combination 

6 

Other 

1 

None 

__4 

Total 

52 

X2(6)  = 

11 

.138 

2 
authors  feel  that  when  the  degrees  of  freedom  of  a  x   test 

2 
are  greater  than  one  the  x   test  should  be  used  only  if  fewer 

than  twenty  percent  of  the  category  cells  have  an  expected 

frequency  of  less  than  five  and  if  none  has  an  expected 

frequency  of  less  than  one  [Ref.  14:p.  110].   However, 

Lancaster  notes  that 

it  used  to  be  feared  by  authors  that  the  test  would 
not  be  accurate  if  carried  out  when  any  class  expec- 
tation was  less  than  10.   These  fears  have  proved 
unjustified  by  the  investigations  of  authors  using 
simulation  methods  or  complete  enumeration  of  all 
possibilities  in  given  discrete  distributions.   [Ref.  15 :p.  175] 

While  Lancaster  does  go  on  to  concede  that  it  is  "probably 

desirable"  to  have  all  expectations  greater  than  one,  he  does 

not  commit  himself  to  making  any  concrete  ruling  on  the  matter. 

2 

As  a  check  on  the  above  test,  a  x   test  was  also  conducted 

with  a  collapsed  contingency  table  comprised  of  row  categories: 
"Original",  "Review"  and  "Other".   A  P  value  of  0.105  was 
obtained,  reinforcing  the  initial  failure  to  reject  the  null 
hypothesis . 

E.   TYPE  OF  WORK  DONE  BY  FUNCTIONAL  AREA 

In  addressing  the  question  of  the  relationship  of  the  type 

2 
of  most  frequent  OR  work  done  and  function  area,  a  x   test  of 

independence  was  again  used.   However,  when  a  contingency  table 
was  set  up  utilizing  all  of  the  categories  in  questions  3  and 
4  of  the  questionnaire,  it  was  found  that  63%  of  the  cell  ex- 
pected frequencies  were  less  than  one.   Since  this  seemed  to 
be  stretching  the  rule-of -thumb  for  cell  minimum  expected 
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frequencies  a  bit  too  far,  the  categories  were  collapsed  to 
those  shown  in  Table  VIII.   The  hypothesis  that  there  is  no 
difference  between  individuals  employed  in  each  of  the  listed 
function  areas  based  upon  the  type  of  OR  work  their  billets 
require  them  to  do  the  most  was  rejected  (P  =  0.0005) . 

TABLE  VIII 
FUNCTIONAL  AREA  VERSUS  TYPE  OF  OR  WORK  DONE 


en  g  en  o 


s  8 


4J 


u        o        o  c  <v  -h  s 

0)            -H            -H  (3  2  -P  0             fl 

Q              01             f0  (U  P  -P            -H 

U  4->  rH  fd 


tj\  Ji  13         ill   10  -H  A  +J 


Original        42     1     5     370     22 
Review  44     3     6     303     23 


Correlate, 
Instructor, 
Combination , 
Other 


0     2     3     0    1    8     16 


Total  10    6     6    14     6    8   11     61 

X2(12)  =  34.92 
P  Value  =  .0005 

F.   COMPUTER  PROGRAMMING 

Only  39.4%  of  the  respondents  reported  that  their  billet 
assignments  required  them  to  do  their  own  computer  programming 
It  was  interesting  to  note  that,  while  44.2%  of  those  who  had 
been  Captains  or  Majors  during  their  most  recent  9  650  tour  re- 
ported being  required  to  do  their  own  programming,  only  21.4% 
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of  those  who  had  been  Lieutenant  Colonels  or  Colonels  reported 
such.   Table  IX  lists  the  reported  frequency  of  programming 
language  use.   Fortran  is  by  far  the  language  most  commonly 
used  by  the  surveyed  analysts,  familiarity  and  availability 
being  the  most  often  reported  reasons  for  its  usage.   In  re- 
gard to  all  of  the  other  listed  languages,  availability  and 
scientific  suitability  of  the  language  were  the  most  often 
reported  reasons  for  usage. 

TABLE  IX 
TYPE  OF  COMPUTER  PROGRAMMING  LANGUAGE  USED 
Language  Frequency  of  Answer   (Percentage! 

Fortran 
Basic 
Mark  IV 
APL 
GPSS 
SPSS 
Cobol 
NIPS 
Total 

A  clear  majority  (66.7%)  of  the  respondents  reported  inter- 
acting with  established  computer  programs  but,  this  time,  a 
higher  percentage  of  Lieutenant  Colonels  and  Colonels  (71.4%) 
reported  using  established  programs  than  did  Captains  and 
Majors  (65.4%) . 
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19 

(38.8) 

9 

(18.4) 

6 

(12.2) 

5 

(10.2) 

3 

(6.1) 

3 

(6.1) 

3 

(6.1) 

_L 

(2.0) 
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(100.0) 

G.   FREQUENCY  OF  USE  OF  OR  TECHNIQUES 

In  question  7  the  respondents  were  asked  to  indicate  the 
frequency  of  usage  of  each  of  35  OR  techniques  and  application 
areas  by  specifying  "Never",  "Sometimes"  or  "Always".   Appen- 
dix B  is  comprised  of  frequency  tables  for  the  answers  obtained 
while  cumulative  frequencies  for  each  subject  are  presented 
in  bar  graphs  in  Figure  1.   Each  of  the  surveyed  techniques 
and  application  areas  is  listed  on  the  horizontal  axes  of  the 
graphs  while  the  vertical  axes  represent  the  percentage  of 
individuals  responding.   The  cross-hatched  section  of  each 
column  indicates  the  percentage  of  personnel  reporting  that 
they  use  the  techniques  "Always"  while  the  clear  enclosed  sec- 
tion reflects  the  percentage  reporting  that  they  use  them 
"Sometimes".   As  an  example,  in  Figure  1  the  graph  shows  that 
21%  of  the  66  respondents  stated  that  they  used  Probability 
Theory  "Always"  and  56%  said  that  they  used  it  "Sometimes". 
Thus,  a  total  of  77%  of  the  sample  reported  using  Probability 
Theory  at  least  sometimes  while  the  remaining  23%  of  the  graph 
is  left  blank,  representing  those  who  responded  that  they  never 
use  that  particular  subject. 

Parametric  Data  Analysis  and  Statistical  Inference,  Cost 
Effectiveness,  Networks  and  Probability  Theory  received  the 
greatest  number  of  responses  for  "Sometimes"  and  "Always"  while, 
surprisingly,  all  of  the  mathematical  programming  techniques, 
including  Linear  Programming,  received  few  usage  responses. 
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H.   IMPORTANCE  OF  OR  TECHNIQUES 

In  question  8  of  the  survey  individuals  were  asked  to  re- 
cord the  relative  importance  to  them  on  a  scale  of  1  =  Extremely 
Important  to  5  =  Not  Important  of  having  a  working  knowledge 
of  the  35  listed  OR  techniques  and  application  areas  in  meeting 
the  daily  requirements  of  their  9650  billets.   The  complete 
listing  of  answer  frequencies  is  contained  in  Appendix  B  while 
Figure  2  includes  bar  graphs  representing  the  cumulative  re- 
sponse frequencies  for  each  subject.   The  surveyed  techniques 
and  application  areas  comprise  the  horizontal  axes  of  the 
graphs  and  the  percentage  of  individuals  responding  are  repre- 
sented by  the  vertical  axes.   The  clear  enclosed  columns  repre- 
sent the  cumulative  percentage  of  individuals  who  marked  a  "1", 
"2"  or  "3"  (extremely  important  to  average  importance)  next  to 
each  subject  while  the  remaining  portion  left  blank  represents 
the  percentage  of  individuals  who  indicated  a  "4"  or  a  "5" 
(little  or  no  importance)  as  their  answer.   For  example,  Figure 
2  shows  that  a  total  of  77%  of  the  66  billet-holders  responding 
said  that  they  felt  that  Probability  Theory  fell  in  the  "1", 
"2"  or  "3"  category  of  their  individual  importance  scales  while 
23%  felt  that  it  held  a  lower  position  on  their  scales  ("4"  or 
"5") .   It  can  be  seen  that,  as  was  the  case  in  usage  frequency, 
Parametric  Data  Analysis,  Probability  Theory,  Cost  Effectiveness 
and  Networks  hold  the  highest  positions  in  an  ordering  of 
importance  based  upon  answers  of  "1",  "2"  and  "3"  while  the 
advanced  mathematical  programming  methods  (Non-linear,  Heuris- 
tic, Dynamic,  Integer  and  Mixed  Programming)  hold  the  lowest. 
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I.   IMPORTANCE  AND  USAGE  FREQUENCY  COMPARISONS 

If  we  take  responses  "1",  "2"  and  "3"  from  question  8  to  mean  Ex- 
tremely Important,  Very  Important  and  Average  Importance,  re- 
spectively, we  can  order  the  listed  techniques  by  importance. 
Combining  "Sometimes"  and  "Always"  from  question  7  also  allows 
us  to  order  these  techniques  by  frequency  of  usage.   We  can, 
then,  compare  order  based  upon  frequency  of  usage  ("Sometimes" 
and  "Always")  (Table  X)  with  order  based  upon  frequency  of 
importance  (average  or  greater  importance)  (Table  XI) .   It 
appears  that  there  is  a  strong  correlation  between  utilization 
and  perceived  importance.   The  only  individual  subjects  that 
do  not  fall  within  a  reasonable  number  of  places  on  both  lists 
are  Equipment  Replacement  and  Transportation  (which  are  higher 
on  the  usage  list  than  on  the  importance  list)  and  Linear 
Algebra  and  Policy  Analysis  (which  are  higher  on  the  importance 
list) .   This  may  indicate  that,  while  Equipment  Replacement  and 
Transportation  are  applied  quite  often,  they  do  not  seem  to 
be  very  important  to  the  using  individuals.   Additionally, 
it  may  be  that,  while  Linear  Algebra  and  Policy  Analysis  may 
not  be  used  very  often,  when  it  does  become  necessary  to  use 
them,  it  is  important  to  know  how  to  use  them.   Another  possible 
explanation  for  the  subjects  which  fall  lower  on  the  usage  list 
than  on  the  importance  list  is  the  imperative  of  knowing  where 
a  technique  does  not  fit.   The  implication  is  that,  knowledge 
of  a  certain  technique  and  knowing  when  not  to  use  it  can  be 
just  as  important  as  knowing  when  to  use  it. 
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TABLE  X 
ORDER  BY  FREQUENCY  OF  USAGE 

Subject  Percentage  (Sometimes  +  Always) 

Data  Analysis  (Parametric)  81 

Cost  Effectiveness  80 

Networks,  Flows  and  Graphs  79 

Probability  Theory  77 

Cost  Estimation  73 
Design,  Conduct  and  Evaluation 

of  Experiments  70 

Data  Analysis  (Nonparametric)  67 

Reliability  62 

Computer  Programming  58 

Human  Factors  53 

Cost  Models  53 

Equipment  Replacement  52 

Economics  51 

Calculus  51 

iMaintenance  and  Repair  51 

Combat  Analysis  49 

Risk  Analysis  45 

Combat  Models  44 

Manpower  Models  42 

Transportation  42 

Linear  Algebra  40 
Queueing  Theory  and 

Stochastic  Models  40 

Gaming  Models  38 

Forecasting  37 

InputHDutput  Models  37 

Linear  Programming  35 

Policy  Analysis  32 

Econometric  Models  28 

Bayesian  Decision  Analysis  27 

Capital  Budgeting  26 

Inventory  Theory  21 

Non- linear  Programming  15 

Dynanic  Programming  14 

Heuristic  Programming  11 

Integer  and  Mixed  Programming  8 
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TABLE    XI 
ORDER   BY    IMPORTANCE 


Subject 

Data  Analysis  (Parametric) 

Probability  Theory 

Cost  Effectiveness 

Networks ,  Flows  and  Graphs 

Computer  Programming 

Data  Analysis  (Nonparametric) 

Reliability 

Design,  Conduct  and  Evaluation 

of  Experiments 
Cost  Estimation 
Cost  Models 
Economics 
Calculus 
Linear  Algebra 
Human  Factors 
Risk  Analysis 
Combat  Analysis 
Forecasting 
Policy  Analysis 
Maintenance  and  Repair 
Manpower  Models 
Gaming  Models 
Linear  Programming 
Equipment  Replacement 
Queueing  Theory  and 

Stochastic  Models 
Combat  Models 

Econometric  Models 
Capital  Budgeting 
Transportation 
Input-Output  Models 
Bayesian  Decision  Analysis 
Inventory  Theory 
Non-linear  Programming 
Integer  and  Mixed 

Programming 
Heuristic  Programming 
Dynamic  Programming 


Percentage  (Categories  1  +  2+3) 

77 
76 
74 
73 
70 
68 
67 


64 
64 
61 
59 
58 
58 
56 
55 
50 
47 
47 
47 
45 
45 
44 
44 

44 
41 
38 
38 
38 
35 
33 
29 
21 

18 
17 
15 


43 


The  association  between  importance  and  utilization  of  OR 
techniques  was  measured  using  the  Spearman  rank  correlation 
coefficient.   For  the  two  lists  in  Tables  X  and  XI  a  rank 
correlation  coefficient  of  +0.918  was  computed,  indicating  a 
highly  positive  correlation.   Testing  the  null  hypothesis  that 
there  is  no  relation  in  the  reported  order  of  usage  frequency 
and  the  reported  order  of  importance  frequency  of  these  OR 
subjects  resulted  in  a  P  value  of  nearly  zero,  indicating 
that,  at  almost  any  level  of  significance,  the  null  hypothesis 
of  no  relationship  would  be  rejected. 

J.   USAGE  BY  OFFICER  RANK 

A  comparison  of  usage  frequencies  was  also  analyzed  by 
seniority  of  rank.   Appendix  C  contains  frequency  tables  based 
on  reported  usage  of  subjects  for  Cap tains /Majors  and  Lieutenant 
Colonels/Colonels.   Figure  3  shows  comparisons  between  Captains/ 
Majors  and  Lieutenant  Colonels/Colonels  for  reported  usage 
frequencies  for  each  of  the  listed  OR  techniques  and  applica- 
tion areas.   The  cross-hatched  column  over  each  subject  indi- 
cates the  percentage  of  Captains  and  Majors  who  reported  using 
that  subject  either  "Sometimes"  or  "Always"  and  the  clear  en- 
closed column  indicates  the  percentage  of  Lieutenant  Colonels 
and  Colonels  who  reported  using  each  subject  "Sometimes"  or 
"Always". 

To  determine  the  greater  frequency  of  usage  for  individual 
techniques,  two  methods  were  utilized,  the  first  method  being 
an  absolute  difference  technique.   Because  of  the  small  number 
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of  personnel  who  reported  having  held  a  9650  billet  as  a 
Lieutenant  Colonel  or  Colonel  (14)  a  change  in  the  answer  of 
one  of  these  men  (e.g.,  from  "Sometimes"  to  "Always")  would 
mean  a  frequency  difference  of  more  than  7%  for  any  particular 
subject.   Therefore,  an  arbitrary  absolute  difference  (. |A|) 
of  10%  was  chosen  to  indicate  a  "significant"  difference  in 
frequency  of  reported  usage  ("Sometimes"  plus  "Always")  for 
each  of  the  two  groups.   Thus,  the  service  rank  group  that 
showed  a  value  for  |a|  of  10%  or  more  in  reported  usage  fre- 
quency for  any  particular  subject  was  deemed  to  have  the  greater 
reported  usage  frequency  for  that  subject.   The  results  obtained 
by  this  method  are  shown  in  Table  XII. 

The  second  comparison  method  was  a  relative  difference 
approach.   Reported  frequencies  ("Sometimes"  plus  "Always")  for 
Captains/Majors,  k,  and  for  Lieutenant  Colonels/Colonels,  I, 
were  used  to  compute  relative  frequencies,  k/£.   The  values 
of  k/fc  above  1.25  (arbitrary)  were  judged  to  mean  a  higher 
usage  frequency  for  Captains  and  Majors  and  those  below  0.75 
were  judged  as  higher  frequencies  for  Lieutenant  Colonels  and 
Colonels.   The  results  obtained  by  this  method  are  reflected 
in  Table  XIII. 

Subjects  that  were  indicated  as  being  used  with  greater 
frequency  by  each  rank  group  by  both  methods  are  shown  in 
Table  XIV.   Two  conclusions  that  might  be  drawn  from  this 
analysis  are  that  it  appears  that  the  more  junior  officers  are 
using  economic  theory  and  techniques  (as  well  as  Computer 
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TABLE  XII 

GREATER  FREQUENCY  OF  USAGE  BY  OFFICER  RANK 
(ABSOLUTE  DIFFERENCE) 


Cap  tains/Ma  j  ors 

Lieutenant  Colonels/ColoneJ 

Ls 

Technique 

|a|% 

Technique 

A|% 

Forecasting 

29 

Capital  Budgeting 

31 

Economics 

28 

Policy  Analysis 

23 

Computer  Prograrmiing 

27 

Risk  Analysis 

17 

Econometric  Models 

17 

Cost  Effectiveness 

16 

Linear  Algebra 

14 

Combat  Models 

16 

Cost  Models 

13 

Data  Analysis 

Ifetworks,  Flows  and 

(Nonparame trie ) 

15 

Graphs 

10 

Gaming  Models 

15 

Probability  Theory 

11 

Combat  Analysis 

11 

TABLE    XIII 

GREATER   FREQUENCY   OF    USAGE    BY   OFFICER   RANK 
(RELATIVE    DIFFERENCE) 


Captains/Majors 

Technique 

Forecasting 

Econometric  Models 

Economics 

Computer  Programming 

Heuristic  Programming 

Linear  Algebra 

Cost  Models 


Lieutenant  Colonels/Colonels 


k/£ 

Technique 

k/i 

3.071 

Capital  Budgeting 

0.380 

2.214 

Integer  and  Mixed 

2.000 

Programming 

0.429 

1.750 

Policy  Analysis 

0.540 

1.714 

Dynamic  P:rogranining 

0.571 

1.483 

Nonlinear  Programming 

0.619 

1.302 

Gaming  Models 

0.700 

Risk  Analysis 

0.702 

Combat  Models 

0.719 

TABLE    XIV 


GREATER   FREQUENCY   OF    USAGE    BY    OFFICER   RANK 
(BOTH  METHODS) 


Cap  tains  /Majors 

Technique 

Forecasting 
Economics 
Econometric  Models 
Computer  Programming 
Linear  Algebra 
Cost  Models 


Lieutenant  Colonels/Colonels 

Technique 

Capital  Budgeting 
Policy  Analysis 
Risk  Analysis 
Gaming  Models 
Combat  Models 
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Programming  and  Linear  Algebra)  more  often  than  their  seniors 
are  and  that  the  senior  officers  are  more  involved  with  the 
budgeting  of  funds  and  the  development  of  policy  than  are  the 
junior  officers  (as  one  might  expect) . 

K.   BREADTH  OF  USAGE  BY  OFFICER  RANK 

Having  studied  the  differences  between  service  rank  groups 
in  regard  to  usage  frequency  of  individual  OR  techniques  and 
application  areas,  we  now  look  at  whether  there  is  a  difference 
between  the  two  officer  groups  based  upon  overall  breadth  of 
usage  of  the  listed  subjects.   An  index  of  breadth  of  usage  was 
created  by  computing  an  average  usage  score  for  each  individual 
in  the  following  manner:   a  value  of  1,  2  or  3  was  assigned  to 
each  box  marked  "Always",  "Sometimes"  or  "Never",  respectively, 
in  question  7  and  these  were  summed  over  all  techniques  and 
application  areas.   This  sum  was  then  divided  by  35,  the  total 
number  of  OR  subjects  listed.   For  example,  an  individual 
questionnaire  with  question  7  having  10  subjects  marked  as 
being  used  "Always",  15  subjects  marked  as  being  used  "Some- 
times" and  10  subjects  marked  as  being  used  "Never"  would  re- 

1+.  •                          -  (10-1)  +  (15-2)  +  (10-3)    -  n 
suit  in  an  average  usage  score  of ■ -rr-= ■ =  2.0 

for  that  particular  individual. 

The  complete  list  of  all  individual  average  scores  was 

divided  into  two  independent  samples  based  upon  service  rank 

(Captains/Majors  and  Lieutenant  Colonels/Colonels) .   A  Kolmo- 

gorov-Smirnov  (K-S)  two-sample  test  was  set  up  to  test  the 

null  hypothesis  that  there  is  no  difference  between  Captains/Majors 
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and  Lieutenant  Colonels/Colonels  based  upon  breadth  of  usage 
of  the  OR  techniques  and  application  areas  listed  in  question 
7  of  the  survey.   Simply  stated,  a  K-S  test  determines  if  the 
cumulative  distributions  of  the  two  samples  are  sufficiently 
divergent  to  suggest  that  the  samples  come  from  different 
populations . 

The  sizes  of  the  samples  in  this  case  varied  from  relatively 
large  (52  Captains  and  Majors)  to  relatively  small  (14  Lieu- 
tenant Colonels  and  Colonels) .   Since  adequate  statistical 
tables  for  the  K-S  test  statistic  were  not  available  for  a 
mixture  of  large  and  small  samples,  two-tailed  tests  for  both 
small  and  large  samples  were  conducted  resulting  in  acceptance 
of  the  null  hypothesis. 

L.   IMPORTANCE  BY  OFFICER  RANK 

Examination  of  these  same  3  5  subjects  with  regard  to  the 
perceived  importance  to  the  two  service  rank  groups  (Captains/ 
Majors  and  Lieutenant  Colonels/Colonels)  resulted  in  the  fre- 
quency tables  presented  in  Appendix  D.   Figure  4  exhibits  com- 
parisons for  each  of  the  listed  techniques  and  application 
areas  for  these  two  groups.   The  cross-hatched  column  over 
each  subject  indicates  the  percentage  of  Captains  and  Majors 
who  reported  an  importance  level  of  "1",  "2"  or  "3"  (average 
or  more  importance)  for  that  subject  and  the  clear  enclosed 
columns  denote  the  percentage  of  Lieutenant  Colonels  and 
Colonels  who  reported  an  importance  level  of  "1",  "2"  or  "3" 
for  each  subject. 
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Absolute  and  relative  difference  methods  similar  to  those 
used  to  determine  frequency  of  usage  differences  were  used  to 
determine  "significant"  differences  between  the  two  officer 
rank  groups  regarding  perceived  importance  of  each  OR  tech- 
nique.  For  the  absolute  difference  method,  it  was  decided 
that  the  service  rank  group  displaying  a  10%  or  greater  abso- 
lute difference,  (|A|),  in  cumulative  frequency  of  average-or- 
more-importance  answers  ("1"  plus  "2"  plus  "3")  for  any  listed 
OR  technique  would  be  sufficient  to  indicate  that  that  subject 
was  regarded  as  more  important  by  one  of  the  officer  ranks. 
Table  XV  reflects  the  results  of  this  procedure . 

TABLE  XV 

GREATER  IMPORTANCE  BY  OFFICER  RANK 
(ABSOLUTE  DIFFERENCE) 

Cap tains /Majors  Lieutenant  Colonels/Colonels 

Technqiue  [A_[%      Technique  [Aj% 

Nonlinear  Programming  18  - 

Dynamic  Programming  17 
Integer  and  Mixed 

Programming  14 

Calculus  10 

Equipment  Replacement    11 

For  the  relative  difference  approach,  ratios  of  cumulative 
frequencies  ("1"  plus  "2"  plus  "3")  for  Captains/Majors  over 
Lieutenant  Colonels/Colonels  (k/£)  were  computed  for  each  of 
the  listed  OR  techniques.   Those  techniques  having  a  k/£ 
ratio  of  1.25  or  higher  were  considered  to  be  of  greater 
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Maintenance  and  Repair 

24 

Econometric  Models 

21 

Cost  Estimation 

17 

Forecasting 

15 

Networks,  Flows  and 

Graphs 

12 

Reliability 

12 

Economics 

12 

importance  to  the  Captains  and  Majors.   Those  techniques  dis- 
playing a  ratio  of  0.75  or  lower  were  deemed  to  be  of  greater 
importance  to  the  Lieutenant  Colonels  and  Colonels.   The 
results  obtained  by  using  this  method  are  displayed  in  Table 
XVI. 

TABLE  XVI 

GREATER  IMPORTANCE  BY  OFFICER  RANK 
(RELATIVE  DIFFERENCE) 


Captains/Majors 

Lieutenant  Colonels/Colonels 

Technique 

k/£ 

Technique              k/l 

Econometric  Models 

2.000 

Dynamic  Programming      0.414 

Maintenance  and  Repair 

1.857 

Nonlinear  Programming     0.500 

Inventory  Theory 

1.429 

Integer  and  Mixed 

Forecasting 

1.429 

Programming           0 . 517 

Transportation 

1.429 

Cost  Estimation 

1.340 

Input-Output  Models 

1.321 

Equipment  Replacement 

1.314 

Subjects  which  were  included  on  both  lists  are  shown  in 
Table  XVII.   It  appears  from  this  table  that  economic  tech- 
niques and  equipment-related  applications  are  perceived  to  be 
more  important  to  the  Captains  and  Majors  while  the  advanced 


TABLE  XVII 

GREATER    IMPORTANCE    BY    OFFICER   RANK 
(BOTH   METHODS) 


Captai  ns/Majors 

Technique 

Cost  Estimation 
Econometric  Models 
Forecasting 
Equipment  Replacement 
Maintenance  and  Repair 


Lieutenant  Colonels/Colonels 

Technique 

Nonlinear  Programming 

Dynamic  Programming 

Integer  and  Mixed  Programming 
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mathematical  techniques  are  held  in  higher  esteem  by  the  more 
senior  officers. 

M.   BREADTH  OF  IMPORTANCE  BY  OFFICER  RANK 

Next,  the  question  of  a  difference  between  the  two  service 
rank  groups  based  upon  perceived  importance  to  the  individuals 
of  the  entire  list  of  OR  techniques  (i.e.,  the  "breadth  of 
importance")  was  addressed.   An  index  of  the  breadth  of 
importance  was  created  and ,  once  again ,  a  Kolmogorov-Smirnov 
two-sample  test  was  used.   An  average  importance  score  for 
each  individual  was  computed  by  assigning  a  value  of  1,  2,  3, 
4  or  5  to  each  box  marked  "1",  "2",  "3",  "4"  or  "5"  in  question 
8,  then  summing  the  values  over  all  techniques  listed  and 
dividing  by  35.   The  average  importance  scores  were  then 
divided  into  two  groups  based  on  the  service  rank  of  each 
individual  and  a  two-tailed  K-S  test  for  two  independent  sam- 
ples was  conducted  resulting  in  acceptance  of  the  null  hypothe- 
sis.  The  implication  of  this  test  is  that  there  is  no  differ- 
ence between  Captains/Majors  and  Lieutenant  Colonels/Colonels 
in  regard  to  their  perceived  feelings  of  importance  of  all  of 
the  listed  OR  techniques  and  application  areas  as  a  whole . 

N.   BREADTH  OF  USAGE  BY  FUNCTIONAL  AREA 

Having  found  no  differences  in  either  overall  breadth  of 
usage  or  overall  breadth  of  importance  of  the  listed  subjects 
based  upon  the  seniority  of  the  respondents,  the  association 
of  breadth  of  usage  and  functional  area  of  the  respondents  was 
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next  inspected.   Respondents'  average  usage  scores,  as  deter- 
mined by  the  method  described  above,  were  partitioned  into  the 
following  general  function  areas:   "Manpower",  "Logistics", 
"Operations  and  Plans",  "Test  and  Evaluation",  "Aviation", 
"Combination"  and  "Other".   A  Kruskal-Wallis  one-way  analysis 
of  variance  was  then  employed  to  test  the  null  hypothesis: 
there  is  no  difference  between  overall  breadths  of  usage  of  the 
OR  techniques  for  individuals  employed  in  the  above-listed  general 
function  areas.   The  Kruskal-Wallis  one-way  analysis  of  variance 
was  chosen  for  use  because  it  requires  only  that  the  variable 
measurement  be  (at  least)  ordinal  and  that  the  variable  have 
an  underlying  continuous  distribution.   In  addition,  the  Kruskal- 
Wallis  test  is  more  efficient  than  some  other  nonparametric 
tests  of  population  differences  because  it  uses  more  of  the 
information  in  the  observations  by  preserving  the  magnitude  of 
the  individual  scores  [Ref.  14:pp.  193-194].   Utilizing  this 
test  and  correcting  for  the  effect  of  ties  in  the  observations 

[Ref.  14:p.  188],  a  test  statistic  of  13.52  was  computed.   This 

2 

statistic  is  approximately  x   distributed  with  6  degrees  of 

freedom  and  yields  a  P  value  of  0.035.   Thus,  the  null  hypothe- 
sis is  rejected,  meaning  that  there  is  a  difference  between 
individuals  (when  they  are  grouped  according  to  the  functional 
area  of  their  billets)  based  upon  their  overall  usage  of  all 
of  the  listed  OR  techniques. 

Close  examination  of  the  means  of  the  average  usage  scores  for 
each  function  area  (displayed  in  Table  XVIII)  may  help  us  under- 
stand a  possible  cause  for  this  null  hypothesis  rejection.   The 
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TABLE  XVIII 
FUNCTION  AREA  MEAN  USAGE  SCORES 


Function  Area 

Mean  Usage 

Score 

Sample 

Size 

Combination 

2.222 

8 

Aviation 

2.405 

6 

Manpower 

2.439 

11 

Test  and  Evaluation 

2.472 

14 

Operations  and 

Plans 

2.543 

7 

Logistics 

2.589 

8 

Other 

2.590 

12 

mean  of  the  average  usage  scores  for  each  function  area  may  be 
interpreted  to  represent  a  measure  of  the  overall  breadth  of 
usage  of  the  listed  subjects  by  individuals  employed  in  each 
of  these  general  areas.   The  closer  a  group's  mean  usage  score 
is  to  1.0,  the  higher  the  overall  frequency  of  usage  of  OR 
techniques  by  the  individuals  in  that  functional  area  (recall 
that  "Always"  =  1,  "Sometimes"  =  2  and  "Never"  =  3) .   There 
is  a  notable  difference  in  mean  breadth  of  usage  for  the 
groups,  especially  in  the  case  where  individuals  worked  in  a 
combination  of  functional  areas.   Intuitively,  we  might  expect 
that  individuals  working  in  a  combination  of  functional  areas 
would  be  apt  to  use  OR  techniques  more  than  those  whose  work 
might  be  channeled  into  one  particular  area. 

0.   BREADTH  OF  IMPORTANCE  BY  FUNCTIONAL  AREA 

The  same  method  as  described  in  the  previous  section  was 
employed  to  test  for  differences  of  breadths  of  importance  of 
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the  subjects  to  individuals  employed  in  the  function  areas 
listed  above.   A  P  value  of  0.0  36  was  obtained,  thus,  again, 
causing  rejection  of  the  null  hypothesis  of  no  difference. 
Looking  at  the  mean  importance  scores  for  each  function  area 
(displayed  in  Table  XIX)  helps  to  explain  this  rejection.   A 
low  score  implies  a  high  overall  perception  of  importance  of 
the  listed  subjects  while  a  high  score  implies  a  low  overall 
feeling  of  importance  (recall  that  Extremely  Important  =  1  and 
Not  Important  =  5) .   Once  again  we  see  a  high  variance  of  mean 
scores,  ranging  from  2.6  36  for  those  individuals  working  in  a 
combination  of  functional  areas  to  3.912  for  those  working  in 
the  field  of  Logistics,  and  again,  intuitively,  we  might  expect 
that  those  individuals  whose  work  is  not  confined  to  a  particu- 
lar area  would  be  more  apt  to  feel  that,  overall,  OR  techniques 
are  more  important  than  those  individuals  restricted  to  working 
in  a  specific  field. 

TABLE  XIX 

FUNCTION  AREA  MEAN  IMPORTANCE  SCORES 

Function  Area  Mean  Importance  Score     Sample  Size 

Combination 

Manpower 

Aviation 

Test  and  Evaluation 

Other 

Operations  and  Plans 

Logistics 


2.636 

8 

3.301 

11 

3.357 

6 

3.400 

14 

3.500 

12 

3.845 

7 

3.912 

8 
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P.   BREADTH  OF  USAGE  BY  TYPE  OF  OR  WORK  DONE 

The  same  procedures  as  described  above  were  used  to  test 
for  differences  between  overall  breadth  of  usage  and  overall 
breadth  of  importance  based  upon  the  following  types  of  OR 
work  that  the  respondents  reported  doing  the  most: 
Original  OR  work 

Reviewing  OR  work  done  outside  their  section 
Any  combination  of  the  types  of  OR  work  listed  in  question 

4  of  the  survey 
Any  other  type  of  OR  work 
None. 
The  average  usage  scores  for  all  respondents  were  divided  up 
into  these  five  groups  and  a  Kruskal-Wallis  one-way  analysis 
of  variance  test  was  conducted  for  the  following  null  hypothe- 
sis:  there  is  no  difference  in  breadth  of  usage  of  OR  techniques 
for  individuals  required  to  do  the  above-stated  types  of  OR  work 
the  most.   A  test  statistic  of  12.36  was  obtained  (having  cor- 
rected for  tied  observations)  resulting  in  a  P  value  of  0.015, 
causing  a  rejection  of  the  null  hypothesis.   This  means  that 
there  is  a  difference  in  breadth  of  usage  of  OR  techniques  be- 
tween individuals  doing  different  types  of  OR  work.   It  can  be 
seen  from  Table  XX  that  mean  usage  scores  varied  from  2.378 
(relatively  high  usage  of  the  techniques)  for  the  group  doing 
original  OR  work  to  2.743  (relatively  low  usage  of  the  tech- 
niques) for  the  group  doing  no  OR  work.   This  would  seem  to 
intuitively  support  the  statistical  findings  of  the  test. 
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TABLE  XX 
MEAN  USAGE  SCORES  BY  TYPE  OF  OR  WORK  DONE 


Mean  Usage 

Score 

Sample  Size 

2.378 

22 

2.4 

10 

2.513 

23 

2.567 

6 

2.743 

5 

Type  of  OR  Work 
Original 
Other 
Review 
Combination 
None 


It  is  interesting  to  note  that,  while  five  individuals 
reported  in  question  4  that  they  did  not  do  any  OR  work,  they 
went  on  in  question  7  to  report  that  they  used  at  least  some 
of  the  listed  OR  techniques  either  "Sometimes"  or  "Always". 

Q.   BREADTH  OF  IMPORTANCE  BY  TYPE  OF  OR  WORK  DONE 

The  Kruskal-Wallis  test  was  also  employed  to  test  for  no 
difference  of  breadth  of  importance  of  the  listed  subjects 
for  the  individuals  required  to  do  the  above-listed  types  of 
OR  work  the  most.   A  P  value  of  0.068  was  obtained,  causing 
null  hypothesis  acceptance  at  the  0.05  level.   This  means 
that,  while  individuals  may  use  the  OR  techniques  to  varying 
degrees  based  upon  the  type  of  work  they  do,  there  is  not  a 
significant  degree  of  disagreement  between  these  individuals 
based  upon  the  overall  perceived  importance  of  knowing  how 
to  use  (and  when  not  to  use)  these  techniques. 
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R.   ADDITIONAL  OR  TECHNIQUES 

Numerous  OR  techniques  and  general  subject  areas  were 
mentioned  by  the  respondents  when  asked  what  techniques  they 
used  in  addition  to  those  listed  in  the  questions.   These  sub- 
jects can  be  separated  into  four  general  categories: 

Management  and  Communications 

Problem  Solving  Procedures 

System  Development 

Computer  Science 

Twenty-five  people  expressed  an  importance  for  knowledge 
in  the  areas  of  Management  and  Communications.   (Management 
techniques,  contract  management,  general  decision  analysis, 
bureaucratic  realities,  management  information  systems,  public 
speaking,  technical  report  writing,  communications  systems, 
salesmanship,  information  theory  and  data  communications  were 
a  few  of  the  subjects  specifically  mentioned.)   Fifteen  indi- 
viduals listed  subjects  that  might  be  incorporated  under  the 
general  heading  of  Problem  Solving  Procedures.   These  included 
problem  definition,  data  collection,  quality  control,  data 
processing  fundamentals,  measures  of  effectiveness,  scaling 
techniques  and  common  sense.   Subjects  possibly  encompassed  by 
the  title  System  Development  were  mentioned  by  12  respondents. 
These  subjects  included  system  design,  system  acquisition,  the 
Planning,  Programming  and  Budgeting  System  and  systems  analysis 
Finally,  9  individuals  felt  that  specialized  knowledge  in  the 
field  of  computers  was  of  great  assistance  in  aiding  them  in 
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the  performance  of  their  duties.   Specifically,  they  listed 
computer  modeling,  computer  program  design,  computer  applica- 
tions and  computer  system  design  as  being  important  to  them. 

S.   TIME  SPENT  DOING  OR  WORK 

When  asked  what  percentage  of  their  work  they  actually  spend 
doing  OR  work  in  comparison  to  their  other  duties,  the  personnel 
surveyed  responded  as  shown  in  Table  XXI.   Overall,  it  appears 
that  the  great  majority  of  those  answering  (78.9%)  spend  twenty- 
five  to  seventy-five  percent  of  their  time  in  the  actual  con- 

2 
duct  of  Operations  Research.   In  addition,  using  a  x   test  for 

two  independent  samples,  the  null  hypothesis  that  there  is  no 

difference  between  Captains/Majors  and  Lieutenant  Colonels/ 

Colonels  based  upon  percentage  of  work  time  actually  spent  doing 

OR  work  was  tested.   The  resultant  P  value  of  0.471  indicates 

acceptance  of  the  null  hypothesis,  indicating  that  there  is  not 

a  significant  difference  between  the  amount  of  work  time- spent 

by  the  more  senior  officers  in  performing  OR-related  tasks 

TABLE   XXI 
PERCENT   OF    WORK    TIME   ACTUALLY    SPENT   DOING    OR  WORK 
Percent      Capt a  i  ns/Ma j ors      Lieutenant  Colonels/Colonels  Total 


0 

8  (15.4) 

25 

12  (23.1) 

50 

12  (23.1) 

75 

16  (30.8) 

100 

4   (7.7) 

Totals 

52  (100) 

Ave.  Time 
Spent 

54.3% 

2 

(14, 

.3) 

5 

(35. 

,7) 

5 

(35, 

.7) 

2 

(14. 

.3) 

0 

(0. 

.0) 

14 

(100) 

3" 

r.5% 

10 

(15.1) 

17 

(25.8) 

17 

(25.8) 

18 

(27.3) 

4 

(6.1) 

66 

(100) 

45.8% 
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and  the  amount  of  work  time  spent  by  Captains  and  Majors  in 
these  types  of  endeavors. 

The  percentage-of-work-time-spent  data  was  also  partitioned 

into  groups  based  upon  general  function  area  as  shown  in  Table 

2 
XXII.   While  it  is  true  that  a  x   test  should  not  be  performed 

on  this  partitioning  due  to  the  fact  that  23%  of  the  expected 
frequencies  fall  below  1.0  and  all  are  below  5.0,  the  wide 
variance  of  average  time  spent  by  individuals  employed  in  each 
function  area  (from  12.5%  for  those  involved  in  Logistics  to 
78.1%  for  those  placing  their  jobs  in  the  category  of  a  combina- 
tion of  function  areas)  suggest  that  there  is  a  difference  be- 
tween individuals  employed  in  each  of  the  listed  function 

TABLE  XXII 
FUNCTIONAL  AREA  VERSUS  PERCENT  OF  WORK  TIME  SPENT  DOING  OR  WORK 

en  a  8 

w  c   en  o  -H 

u  o  £         "?  ".3  S  la 

QW  fd  P-.  P  +J  -H  U  H 

S  gi  i'S03^"^  S  5  o 

s5  a§BW<  o  o  p 

0%  2430001  10 

25%  1           4              2              3              2              0              5  17 

50%  5           0              1              5              3              1              2  17 

75%  3            0              1              4              1              5              4  18 

100%  _?_P_P._i.P_2__0  _4 

Total  11           8              7            14              6              8            12  66 

Average 

Time        45.5     12.5     25.0      58.9      33.3      78.1      43.75 
Spent 
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areas  based  upon  the  percentage  of  work  time  actually  spent 
doing  OR  work. 

Of  the  27  individuals  who  reported  spending  25%  or  less 
of  their  work  time  performing  explicit  OR  functions,  9  stated 
that  the  majority  of  their  time  was  spent  performing  general 
management  functions,  7  reported  that  studies  review  or  policy 
review  (that  did  not  require  explicit  OR  work)  consumed  most 
of  their  time,  4  reported  administrative  duties  and  3  said 
working  with  computers  kept  them  busy  most  of  the  time. 

T.   ACCEPTANCE  OF  OR  WORK  BY  SUPERIORS 

Table  XXIII  contains  the  results  of  the  response  when 
individuals  were  asked  to  relate  the  basic  level  of  acceptance 
of  OR  work  by  their  superiors.   As  can  be  seen,  the  majority 

reported  that  OR  work  is  usually  accepted  while  none  of  the 

2 

respondents    said   that  OR  work   is   never    accepted.      A   x      test 

was    used    to   test   the   null   hypothesis    that    there    is   no 

TABLE    XXIII 
OR   WORK   ACCEPTANCE    LEVEL 

Acceptance  Level              Captai  ns/Majors  Lt.  Colonels/Colonels  Total 

Accepted  w/o  Question         10    (20.4)  4  (28.6)  14  (22.2) 

Usually  Accepted                   35    (71.4)  9  (64.3)  44  (69.8) 

Rarely  Accepted                      4      (8.2)  1  (7.1)  5  (7.9) 

Never  Accepter                        0     (0.0)  0  (0.0)  0  (0.0) 

Totals                                        49    (100)*  14  (100)  63  (100)* 

* 
Note:     3  individuals  failed  to  answer  the  question. 
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difference  between  Captains/Majors  and  Lieutenant  Colonels/ 
Colonels  based  upon  the  acceptance  level  of  OR  work  of  those 
for  whom  they  work.   (The  category  of  "Never  accepted"  was 
dropped  from  the  test  since  the  number  of  observations  from 
both  samples  for  this  category  was  zero.)   A  P  value  of  0.81 
was  obtained,  resulting  in  acceptance  of  the  null  hypothesis 
Thus,  it  appears  that  the  acceptance  level  of  OR  work  as 
perceived  by  the  respondents  is  independent  of  the  service 
rank  of  those  proffering  it. 
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IV.   COMMENT  ANALYSIS 

The  analysis  described  in  the  previous  section  relates 
directly  to  the  first  eleven  survey  questions,  all  of  which 
can  be  described  as  being  of  a  quasi-objective  nature.   Ques- 
tion 12,  an  open-ended  solicitation  of  comments,  resulted 
in  billet  utilization  recommendations  which  can  only  be 
characterized  as  subjective.   For  this  reason,  the  results 
obtained  from  question  12  will  be  discussed  in  this  separate 
chapter. 

One  should  bear  in  mind  when  reading  this  analysis  that 
in  no  case  did  a  majority  of  the  respondents  address  any 
single  subject.   Rather,  all  topic  discussions  have  been 
derived  from  the  comments  made  by  two  to,  at  most,  thirteen 
individuals.   The  author's  intent  in  this  chapter  is  to  re- 
late to  the  reader  not  so  much  the  general  consensus  of  the 
entire  sample  population  as  it  is  to  give  the  flavor  and 
substance  of  the  returned  comments.   In  this  manner  it  is 
possible  to  illuminate  some  of  the  underlying  issues  regard- 
ing the  use  of  Operations  Analysts  and,  hence,  to  make  possi- 
ble recommendations  for  policy  changes. 

The  most  enlightening  notion  that  question  12  evoked  was 
that  Operations  Research  is  really  a  way  of  thinking  which 
should  be  required  of  all  Marine  officers.   Many  respondents 
found  their  OR  backgrounds  to  be  almost  as  valuable  to  them 
in  their  assignments  as  line  officers  as  in  their  SEP  billet 
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tours.   They  felt  that  the  real  value  of  their  advanced 
education  was  in  providing  them  with  an  analytical  method 
of  formulating  problems,  thinking  about  causes  and  then 
determining  their  solutions  in  a  systematic  manner.   Pure 
OR  work  is  rare,  they  found,  but  techniques,  such  as  mathe- 
matical programming,  provide  a  good  point  of  origin  for 
thinking  about  specific  problems  (even  though  "a  heuristic/ 
seat-of-the-pants  approach  is  what  is  actually  required"). 

Many  individuals  also  stated  that  there  is  an  apparent 
lack  of  understanding  on  the  part  of  many  senior  officers 
in  managerial  positions  as  to  what  Operations  Research  is 
and  what  analysts  have  been  trained  to  do.   This  unawareness 
often  leads  to  misuse  of  the  analysts1  technical  skills  or, 
worse,  skepticism  as  to  the  validity  of  analytical  work. 
Two  officers  reported  that  OR  is  often  confused  with  Computer 
Science  and,  consequently,  they  spend  much  of  their  time 
writing  and  revising  computer  programs  that  are  not  related 
to  analysis,  while  many  respondents  remarked  upon  the  apparent 
suspicion  among  some  senior  officers  in  the  Corps  when  opera- 
tions analysis  is  suggested  as  a  viable  method  of  problem 
solving.   Classical  Operations  Research  techniques,  such  as 
those  listed  in  the  questionnaire,  are  often  considered  to 
be  esoteric  by  senior  officers  so,  consequently,  9650  billet- 
holders  must  often  solicit  OR  work  and  then  "sell"  their 
results. 

The  source  of  these  problems  seems  to  be  a  rather  deep- 
seated,  historical  one  and  was  succinctly  described  by  one 
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Lieutenant  Colonel  who  wrote: 

OR  analysts  are  viewed  with  mixed  emotions — the 
manipulation  of  analysis  by  Pentagon  officials 
starting  with  McNamara  'Whiz  Kids'  who  set  out 
to  prove  the  bottom  line  .still  persists  in  PA&E 
[Program  Analysis  and  Evaluation] .   [These  offi- 
cials] get  their  marching  orders  from  the  politi- 
cal appointees  in  the  administration.   Military 
analysts  at  that  level  are  used  by  the  services 
to  conduct  counter  studies  or  to  discredit  work 
done  by  the  administration  if  it  is  counter  to 
service-perceived  goals.   We  [Operations  Analysts] 
are  viewed  as  necessary  evils  and  are  tainted  by 
the  misuse  of  OR. 

One  suggested  solution  to  this  dilemma  was  to  educate 
the  officers  filling  supervisory  positions  over  the  analysts. 
In  fact,  it  was  felt  by  many  respondents  that  common,  basic 
OR  education  would  lead  not  only  to  a  greater  degree  of 
acceptance  of  the  work  of  OR  trained  technicians  but  also 
to  a  more  productive  officer  corps  as  a  whole. 

In  regard  to  actual  utilization  of  the  billets,  almost 
twenty  percent  of  the  respondents  suggested  that  the  pro- 
ductivity of  OR  billets  might  be  improved  by  centralizing 
most  of  them  in  a  single  Operations  Analysis  Group.   Being 
spread  throughout  the  Marine  Corps,  as  they  are  now,  many 
analysts  have  found  that  the  low  incidence  of  OR  work  brought 
to  them  often  does  not  justify  their  billets.   This  appeared 
to  be  particularly  true  at  the  Division/Wing  level  where  the 
fast  pace  of  daily  activity  brought  about  by  what  is  per- 
ceived to  be  excessive  demands  often  results  in  quick  prob- 
lem response  at  a  superficial  level  of  analysis.   This  is  not 
conducive  to  the  methodical  and  measured  thought  process  often 
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required  for  thorough  and  accurate  analytical  work.   Conse- 
quently, analysts  at  the  lower  command  levels  often  either 
become  embedded  in  administrative  oblivion  and  structure 
politics  or  they  are  simply  converted  into  efficiency  experts 
assigned  the  task  of  acting  as  command  hatchet-men,  seeking 
out  waste  and  misuse  of  resources. 

Recommendations  for  correcting  this  deficiency  were  either 
to  disestablish  OR  billets  below  the  FMF  level  or,  better, 
to  centralize  them  within  a  single  department.   A  central  sec- 
tion, perhaps  under  the  direction  of  a  Deputy  Chief  of  Staff 
for  Operations  Analysis,  could  accept  viable  projects  submitted 
by  other  departments  or  commands  throughout  the  Corps .   Teams 
of  one  to  six  analysts  (depending  upon  the  size  and  scope  of 
the  problem)  with  appropriate  backgrounds  could  be  assigned 
to  each  project  on  an  as-needed,  as-available  basis.   This 
reorganization  would,  in  the  opinion  of  the  respondents ,_ lead 
to  better  long-range  study  planning  and  a  less  hectic  daily 
press  for  results. 

This  recommendation  goes  counter  to  the  findings  of  Thomas 
and  DaCosta  [Ref.  9].   In  their  survey  of  private  industry 
they  found  that  the  most  recent  trend  in  the  private  sector 
has  been  away  from  centralized  Operations  Research  departments. 
Their  survey  disclosed  that  "many  respondents  felt  that  by 
spreading  Management  Science  [Operations  Research]  throughout 
the  firm,  practitioners  would  be  better  acquainted  with  the 
problems  they  had  to  solve  and,  therefore,  would  be  more 
efficient"  [Ref.  9,  p.  103]. 
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A  number  of  individuals  addressed  the  subject  of  OR  work 
contracted  out  by  the  Marine  Corps  to  civilian  organizations. 
Many  of  the  respondents  expressed  concern  over  the  Marine  Corps' 
apparent  eagerness  to  issue  contracts  to  outside  sources  who 
often  charge  exorbitant  fees  but  who  sometimes  use  incorrect 
OR  techniques  and/or  produce  questionable  results.   One  indi- 
vidual, working  in  the  area  of  Test  and  Evaluation,  felt  that 
"the  Marine  Corps  is  paying  too  much  to  'Beltway  Bandits' 
[civilian  contractors]  and  their  ilk  for  studies  which  are 
nothing  more  than  mumbo  jumbo".   Many  respondents  felt  that, 
if  assigned  to  these  tasks,  they  could  do  most  of  the  techni- 
cal work  themselves  thus  producing  better  results  at  a  fraction 
of  the  costs. 

The  general  idea  expressed  was  that,  in  order  for  Operations 
Analysts  to  be  used  most  effectively,  they  should  be  "doers" 
rather  than  mere  contract  supervisors.   If  resource  limita- 
tions prevent  this,  however  (e.g.,  more  work  than  can  be  done 
by  available  Marine  analysts)  then  the  analysts  should  at 
least  be  used  to  review  the  entire  process  of  contracted 
studies  and  not  just  the  finished  product.   Marine  analysts 
should  be  included  in  the  original  contract  negotiations  with 
civilian  agencies  to  ensure  that  the  Marine  Corps  is  not  led 
into  allowing  meaningless  or  unnecessary  tests  to  be  performed. 
Operations  Analysts  should  monitor  the  contracted  work  as  it 
progresses  and  should,  certainly,  inspect  the  final  results. 
According  to  the  respondents,  this  monitoring  and  review  work 
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must  be  a  comprehensive  quality  control  function.   If  the  role 
filled  by  the  9  650  is  merely  that  of  "riding  herd"  over  previ- 
ously contracted  studies,  then  it  was  felt  that  an  individual 
with  a  non-technical  background  such  as  a  Systems  Analyst 
(MOS  9652)  could  much  more  efficiently  fill  the  billet. 

The  greatest  number  of  comments  received  addressing  any 
single  subject  (13)  were  those  that  recommended  a  reevaluation 
of  the  requirements  for  9650  billets.   While  an  annual  review 
of  all  SEP  billet  requirements  is  conducted  by  HQMC ,  the 
respondents,  generally,  did  not  believe  it  to  be  comprehen- 
sive enough.   Two  main  reasons  were  perceived  by  the  respon- 
dents as  being  responsible  for  the  unnecessary  establishment 
of  billets:   (1)  The  billets  were  originally  requested  and 
approved  by  individuals  who  really  didn't  understand  Operations 
Research  and  its  applications  and  (2)  the  desire  by  some  senior 
officials  to  add  intelligent  (but  possibly  unnecessary)  offi- 
cers to  their  staffs.   One  of  the  results  of  this  billet 
inflation  is  that,  very  often,  a  promising,  career-oriented 
officer  is  assigned  to  a  billet  that  requires  him  to  use  very 
little  of  his  special  training  and  expertise.   In  fact,  so 
much  time  appears  to  be  spent  by  analysts  working  on  special 
but  non-analysis  related  staff  projects  that  some  individuals 
expressed  doubt  as  to  the  need  for  any  OR  billets. 

While  there  are,  of  course,  billets  that  do  require  and 
utilize  a  working  knowledge  of  OR  techniques,  it  was  rather 
obvious  from  the  comments  received  that  a  truly  comprehensive 
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review  of  all  9650  billet  requirements  would  probably  lead 
to  the  realization  that  some  billets  are  either  misused  or 
not  essential.   Once  identified,  these  billets  could  be  dis- 
established or  modified  (e.g.,  replace  a  9650  with  a  9652) 
or  the  cause  of  misutilization  could  be  corrected.   The  real 
problem,  however,  as  viewed  by  those  who  have  held  the  billets, 
would  lie  in  the  identification  process.   This  would  require 
a  frank  and  honest  review  of  the  requirements  for  all  9  650 
billets  and,  while  it  was  generally  felt  that  the  analyst 
holding  an  "unessential"  billet  would  probably  acknowledge 
such,  there  was  little  faith  that  his  command  would  make  the 
same  acknowledgment.   In  fact,  several  individuals  expressed 
concern  that  there  might  be  retribtuion  in  the  form  of  poor 
fitness  reports  if  an  organization  lost  any  billets  due  to 
the  honest  evaluation  of  an  incumbent.   One  respondent  sum- 
marized the  feelings  expressed  by  several  by  writing, 

What  the  Marine  Corps  needs  is  a  truly  covert, 
non-attributable  billet  validation  procedure.   No 
bureaucratic  organization  is  going  to  give  up  a 
body  voluntarily  unless  it  can  get  something  in 
trade  of  equal  or  greater  benefit,  i.e.,  the  organi- 
zation will  always  validate  the  billet.   If  I  am 
scheduled  to  leave  and  my  boss  is  told  by  HQMC 
(He  starts  calling.  .  .  long  before  a  person  is 
actually  scheduled  to  leave,  demanding  .  .  .  replace- 
ment. )   that  they  have  decided  not  to  fill  this 
billet  anymore,  then  his  automatic  inference  is 
that  I  'dropped  the  dime1  and  he  gives  appropriate 
award  on  my  parting  fitness  report.   Hence,  what 
is  needed  is  covert  elicitation  of  billet  vali- 
dation from  incumbents  and  protection  in  those 
cases  where  billets  cannot  be  validated  and  will 
be  eliminated  (e.g.,  stall  and  stonewall  on 
replacement) . 

If  such  a  system  of  annual  validation  could  be  established, 

several  individuals  felt  that  as  many  as  one-third  of  the 
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present  number  of  9650  billets  might  be  made  available  for 
more  productive  use.   (Indeed,  it  was  seen  above  that  almost 
41%  of  the  respondents  stated  that  they  spent  25%  or  less  of 
their  work  time  actually  performing  OR  work.) 

Two  individuals  did  state  that  the  Marine  Corps  needs  more 
9  65  0  billets.   They  declared  that  there  are  many  people  (who 
are  unaware  of  the  value  of  applying  OR  techniques)  attempting 
to  solve  problems  which  could  be  handled  much  more  efficiently 
using  analytical  methods.   Headquarters,  Marine  Corps  apparently 
has  a  virtual  gold  mine  of  OR  solvable  problems  just  waiting 
to  be  recognized  by  someone  versed  in  the  field.   What  is 
needed,  according  to  these  individuals,  is  enough  trained 
analysts  to  ferret  these  problems  out  and  solve  them. 

Finally,  two  individuals  recommended  that  prospective  OR 
graduates  be  paired  with  their  initial  billet  assignments  as 
early  as  practical  (at  least  two  quarters  prior  to  graduation 
and,  ideally,  a  year  prior).   The  current  SEP  monitor  at  HQMC 
realizes  the  importance  to  students  of  early  billet  notifica- 
tion and  is  attempting  to  do  this  now.   Certainly,  this  does 
require  planning  and  foresight  on  his  part  but  unforeseen 
difficulties  do  arise  and  changes  in  prospective  assignments 
must  sometimes  be  made.   Still,  early  notification  does  allow 
selection  of  elective  courses  oriented  toward  the  billet  to 
be  filled  and  permits  proper  preparation  on  the  part  of  the 
students.   These  students  should  bear  in  mind,  however,  that 
subsequent  965  0  tours  might  be  related  to  entirely  different 
function  areas  than  their  initial  billet  assignments;  hence, 
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they  should  not  design  elective  programs  restricted  solely 
to  one  field  but  should  try  to  maintain  at  least  a  minimum 
balance  of  OR  subjects. 

Some  of  the  most  interesting  results  of  the  survey  were 
those  obtained  from  the  answers  to  question  12  as  described 
in  this  chapter.   Although  usually  no  more  than  four  or  five 
of  the  72  responding  individuals  addressed  any  one  particular 
subject,  many  of  the  remarks  appeared  to  be  well  thought-out 
and  reflected  the  advantage  of  being  based  upon  actual  experi- 
ence.  It  should  be  reemphasized,  however,  that,  simply  be- 
cause one,  two  or  even  six  individuals  voiced  an  opinion,  it 
does  not  necessarily  mean  that  their  comments  are  reflective 
of  the  entire  population.   Even  with  this  caveat  of  subjectivity, 
however,  it  is  possible  to  draw  attention  to  the  issues  that 
have  been  raised  and  to  make  subsequent  recommendations  re- 
garding them.   This  is  done  in  the  next  chapter  subsequent 
to  a  short  comparison  of  the  results  of  this  study  with  those 
of  studies  done  previously. 
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V.   COMPARISONS,  CONCLUSIONS  AND  RECOMMENDATIONS 

A.   COMPARISONS  WITH  PREVIOUS  STUDIES 

Since  the  format  and  conduct  of  this  study  did  not  exactly 
match  those  of  investigations  done  previously,  including  Obert's 
survey  of  Army  OR  technique  usage,  the  results  cannot  be 
collated  in  a  parallel  manner.   However,  some  comparisons 
can  be  made . 

Data  analysis  and  statistical  methods  were  the  most  fre- 
quently used  OR  techniques  as  in  all  the  studies  reviewed 
in  Chapter  I.   Linear  Programming  was  reported  to  be  used  by 
only  about  a  third  of  the  Marine  Corps  respondents,  matching 
the  results  found  in  Obert's  study  of  Army  OR/SA  engineers 
but  falling  quite  a  bit  short  of  the  apparent  wide-spread  usage 
reported  by  private  firms  (where  LP-amenable  problems  such  as 
resource  allocation  are  possibly  more  numerous  and  better  de- 
fined) .   Inventory  Theory  and  the  more  advanced  mathematical 
programming  subjects  were  generally  listed  higher  in  frequency 
and  importance  orderings  for  corporate  industry  than  they  were 
in  this  study  while  Risk  Analysis  was  found  to  be  more  widely 
used  by  Marine  analysts  than  by  private  firms  as  reported  by 
Thomas  and  DaCosta.   Once  again,  this  is  probably  a  reflection 
of  the  natures  of  the  two  organizations.   Private  firms  are 
more  concerned  with  the  resource  allocation  and  sales  problems 
while  military  operations  characteristically  require  a  high 
level  of  risk-taking. 
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Some  of  the  most  interesting  and,  perhaps  because  of  the 
nature  of  employment  of  the  persons  surveyed,  some  of  the 
most  appropriate  comparisons  can  be  made  between  this  study 
and  the  one  conducted  by  Obert.   The  author  found  that  a 
smaller  percentage  of  Marines  than  Army  officers:   (1)  Con- 
ducted original  OR  work,  correlated  OR  work  or  reviewed  OR 
work  (74%  to  Obert's  87%).   (2)  Worked  with  established  com- 
puter programs  (66.7%  to  Obert's  80.3%).   (3)  Reported  spend- 
ing 50%  or  less  of  their  work  time  doing  OR  work  (66.7%  to 
Obert's  73.2%).   Also,  unlike  Obert's  study,  this  investiga- 
tion found  that  there  is  not  a  significant  difference  between 
the  amount  of  work  time  spent  by  Captains/Majors  and  the 
amount  of  work  time  spent  by  Lieutenant  Colonels/Colonels  in 
the  performance  of  OR-related  tasks.   Operations  Research 
techniques  which  were  found  to  be  placed  in  nearly  equal 
positions  in  orderings  of  frequency  of  use  by  Marine  analysts 
and  Army  OR/SA  engineers  included  Parametric  Statistics,  Pro- 
bability Theory,  Networks,  Human  Factors,  Linear  Algebra, 
Linear  Programming,  Non-Linear  Programming  and  Dynamic  Pro- 
gramming.  Techniques  which  reflected  a  more  widespread  use 
by  Army  officers  than  by  Marines  included  Combat  Models  and 
Decision  and  Risk  Analysis,  while  the  Marines  reported  greater 
usage  of  Experimental  Design,  Nonparametric  Statistics,  Cost 
Estimation,  Calculus,  Economics,  Manpower  Models  and  Input/ 
Output  Models.   Barring  these  differences,  however,  the  remain- 
der of  the  results  of  this  work  were  relatively  similar  to 
those  of  Obert's  study. 
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B.   CONCLUSIONS 

This  section  includes  the  main  conclusions  drawn  from  the 
analysis  of  the  data  and  comments  obtained  from  the  returned 
questionnaires. 

1.   Data  Analysis 

Of  all  the  OR  techniques  listed  in  the  questionnaire, 
the  more  advanced  mathematical  programming  methods  such  as 
Nonlinear,  Dynamic,  Heuristic,  Integer  and  Mixed  Programming 
are  used  the  least  and  believed  to  be  the  least  important 
by  the  surveyed  analysts  in  carrying  out  their  daily  duties 
as  9650 *s.   Parametric  Data  Analysis,  Probability  Theory,  Cost 
Effectiveness  and  Networks  are  the  subjects  used  the  most  and 
believed  to  be  the  most  important. 

While  it  was  found  that  there  are  differences  in  usage 
of  individual  techniques  according  to  service  rank  groups 
(Captains  and  Majors  use  economic  theory  and  related  techniques 
such  as  Econometric  Models,  Cost  Models  and  Forecasting  as  well 
as  Linear  Algebra  and  Computer  Programming  more  often  than  do 
Lieutenant  Colonels  and  Colonels  while  the  latter  use  Risk 
Analysis,  Gaming  Models,  Combat  Models,  Capital  Budgeting  and 
Policy  Analysis  more  often  than  do  the  former) ,  on  an  overall 
basis  there  is  no  statistical  difference  between  the  two  groups 
based  upon  the  breadth  of  usage  of  all  of  the  OR  techniques 
and  application  areas.   What,  then,  might  be  possible  causes 
for  differences  in  overall  breadth  of  usage? 

It  was  found  that  there  are  statistically  significant 
differences  between  breadths  of  usage  of  the  techniques  based 
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upon  the  general  function  areas  related  to  the  9650  billets 
and  based  upon  the  type  of  OR  work  that  the  analysts  were 
required  to  do  the  most.   As  might  be  expected,  individuals 
working  in  a  combination  of  functional  areas  and  those  per- 
forming original  OR  work  were  found  to  use  a  wider  variety  of 
OR  techniques  than  those  analysts  employed  in  separate  func- 
tional areas  or  those  performing  various  other  types  or  com- 
binations of  OR  work. 

Regarding  perceived  importance  of  the  techniques  and 
application  areas  listed  in  the  questionnaire,  individual 
differences  also  exist  based  upon  the  two  service  rank  groups. 
Economic  models  (Econometric  Models,  Cost  Estimation  and  Fore- 
casting) as  well  as  Equipment  Replacement  and  Maintenance  and 
Repair  are  perceived  to  be  more  important  by  Captains/Majors 
than  by  Lieutenant  Colonels/Colonels  while,  on  the  other  hand, 
most  of  the  advanced  mathematical  programming  methods  (Non- 
linear, Dynamic,  Integer  and  Mixed  Programming)  are  perceived 
to  be  more  important  by  the  Lieutenant  Colonels  and  Colonels. 

Once  again,  however,  no  difference  was  found  to  exist 
between  the  two  service  rank  groups  based  upon  overall  breadth 
of  importance  of  all  the  techniques,  nor  was  a  difference  found 
based  upon  the  different  types  of  OR  work  required  to  be  per- 
formed by  the  individuals.   (This  might  imply  that  the  respon- 
dents placed  a  relative  equality  of  importance  on  the  techniques 
whether  they  used  them  or  not.)   The  only  possible  causes  for 
differences  in  overall  breadth  of  perceived  importance  of  the 
listed  techniques  was  that  attributed  to  variation  in  the  general 
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function  areas  related  to  the  9650  billets.   Those  individuals 
working  in  the  areas  of  Logistics  and  Operations  and  Plans 
assigned  the  lowest  breadth  of  importance  value  to  the  tech- 
niques while  those  working  in  a  combination  of  areas  assigned 
the  highest  breadth  of  importance  value. 

Two  other  determinations  were  made  regarding  the  usage 
and  importance  of  techniques.   First,  there  is  a  strong  corre- 
lation between  reported  utilization  of  and  perceived  importance 
of  the  3  5  OR  techniques  listed  in  the  questionnaire  (which  is 
rather  intuitive — the  more  a  technique  is  used,  the  more  likely 
it  is  that  the  user  will  find  it  to  be  important \ .   Secondly, 
although  most  Marine  analysts  are  not  required  to  write  their 
own  computer  programs,  those  who  do  write  their  own  use  Fortran 
as  their  most  frequent  programming  language.   (It  was  noted 
that  most  analysts  do,  however,  interact  with  established  or 
"canned"  programs) . 

Analysis  was  also  conducted  to  determine  reasons  for 
possible  differences  in  the  type  of  OR  work  required  to  be 
done  by  the  9650' s,  the  percentage  of  work  time  Marine  analysts 
spend  doing  OR  work  and  the  level  of  acceptance  of  OR  work 
by  those  for  whom  the  respondents  worked.   No  differences  in 
any  of  these  three  areas  were  found  based  upon  the  two  service 
rank  groupings  (Captain/Major  and  Lieutenant  Colonel/Colonel) . 
However,  the  type  of  OR  work  performed  was  found  to  vary 
according  to  billet-related  functional  areas.   (It  is  inter- 
esting to  note  that  OR  work  is,  at  least,  usually  accepted  by 


81 


most  superiors  and  in  no  case  was  it  found  to  be  consistently 
not  accepted. ) 

2 .   Comment  Analysis 

It  is  rather  more  difficult  to  draw  hard  and  fast 
conclusions  from  the  comment  analysis  section  than  from  the 
data  analysis  section  of  this  study.   The  subjective  nature 
of  the  comments ,  as  well  as  the  fact  that  none  of  the  topics 
discussed  were  mentioned  by  a  majority  of  the  respondents 
direct  us  to  accept  the  fact  that,  rather  than  producing  con- 
clusions, the  comments  really  simply  raise  issues  that  require 
further  study. 

The  considerable  response  that  was  obtained  when  the 
analysts  were  asked  to  make  recommendations  for  improving  the 
utilization  of  9650' s  appears  to  reflect  at  least  some  measure 
of  job  dissatisfaction  on  the  part  of  the  respondents.   The 
main  cause  of  this  disappointment  is  probably  the  current  mis- 
use of  some  Marine  Corps  analysis  resources.   A  strategy  for 
alleviating  this  misuse  can  be  developed  by  addressing  the 
three  major  issues  brought  out  by  the  comments.   These  are: 
(1)  The  need  for  education  of  965  0  billet  supervisors.   (2)  The 
need  for  education  of  Marine  Operations  Analysts.   (3)  The 
need  for  reevaluation  and  possible  reorganization  of  current 
9650  billets. 

a.   Educating  Supervisors 

It  is  clear  from  the  comments  that  there  is  some 
organizational  resistance  to  the  initiation  and  implementation 
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of  OR  work.   This  was  judged  to  be  due  mainly  to  the  misunder- 
standing on  the  part  of  some  senior  Marine  officers  as  to  what 
OR  really  is  and  what  its  value  can  be  in  the  decision-making 
process.   While  it  is  understandable  that  a  number  of  super- 
visors will  be  unversed  in  the  more  mundane  esoterica  of  Opera- 
tions Research,  it  should  certainly  be  desirable  to  educate 
these  officers  so  that  the  goal  of  proper  utilization  of  analyst 
resources  might  be  attained.   Just  as  an  infantry  officer  should 
be  knowledgeable  in  the  proper  deployment  and  employment  of 
his  troops,  so  should  a  supervisor  of  Operations  Analysts  be 
capable  of  intelligently  assigning  and  employing  his  personnel 
resources . 

A  more  difficult  problem  to  counter  is  that  of  en- 
grained resistance  to  change  and/or  new  ideas  due  to  limited 
vision  or  funneled  perception.   Some  individuals  are  simply 
opposed  to  innovative  but  obscure  approaches  to  problems  simply 
because  their  past  experiences  have  included  only  simple,  "head- 
on"  approaches.   While  it  may  certainly  be  argued  that  in  many 
cases  the  direct  approach  is  the  best  course  of  action,  super- 
visors should  at  least  be  open  to  suggestions  concerning  novel 
methods  of  reaching  problem  solutions. 

b.   Educating  Operations  Analysts 

One  factor  contributing  to  9650  job  dissatisfaction 
is  the  frustration  felt  by  the  analyst  when  he  is  unexpectedly 
confronted  by  the  realities  of  analytical  work  in  the  "real 
world".   This  is  especially  true  in  the  cases  of  young  analysts 
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who  have  recently  finished  their  graduate  work  and  are  zealously 
approaching  their  first  assignments  confident  that  they  are 
capable  of  performing  creditable  analytical  work.   What  they 
are  often  surprised  to  find  is  that  analysis  in  real  life  is 
not  conducted  in  the  same  pristine  and  aseptic  environment 
that  they  often  enjoyed  when  approaching  problems  in  the  class- 
room.  In  fact,  they  may  very  quickly  learn  that  their  particu- 
lar billet  assignment  requires  them  to  do  very  little,  if  any, 
real  analytical  work  or  that  they  might  spend  several  weeks 
on  a  project  only  to  have  the  results  ignored  by  decision- 
makers or,  even  worse,  that  they  might  see  good  analytical 
work  disregarded  for  nontechnical  reasons.   The  prospective 
965  0  should  be  made  aware  of  the  true  nature  of  the  environ- 
ment that  he  is  about  to  enter  prior  to  his  arrival  at  his 
initial  assignment.   He  must  understand  that  he  may  encounter 
institutional  resistance  to  OR  work  and  that,  in  the  real  world 
of  give-and-take,  compromises  must  sometimes  be  made  for- politi- 
cal reasons.   Additionally,  there  will  be  times  when  the  Opera- 
tions Analyst  must  act  as  a  technological  change  agent,  soli- 
citing problems  that  can  be  solved  by  OR  techniques  and  then 
actively  selling  his  results  to  the  decision-makers.   By  edu- 
cating the  analyst  in  regard  to  occupational  realities  it  might 
be  possible  to  mitigate  the  frustrations  he  might  feel  because 
of  unfulfilled  expectations. 

c.   Reevaluation  and  Reorganization  of  Billets 

There  appear  to  be  cases  where  Marine  Operations 
Analysts  are  being  improperly  utilized.   Disenchantment  was 
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expressed  by  the  9650 's  because  of  low  OR  work  incidence, 
limitations  of  available  analytical  resources  (both  personnel 
and  computational  hardware)  and  simple  misunderstandings  con- 
cerning the  capabilities  and  limitations  of  analysis.   They 
felt  that  spreading  OR  billets  throughout  the  Marine  Corps 
can,  in  some  instances,  lead  to  inefficiency  and  misuse  of 
resources  and  that,  at  times,  OR-related  projects  are  unneces- 
sarily contracted  out  by  the  Marine  Corps  to  civilian  agencies 
while  the  "in-house"  analysts  who  could  do  the  work  themselves 
are  assigned  as  Project  Officers  and  relegated  to  a  role  of 
simple  supervision  and  review.   The  opinion  of  many  of  the 
respondents  was  that  9650  billets  are  overdue  for  a  hard, 
honest  reevaluation  of  their  technical  skill  requirements  and 
organizational  locations. 

C .   RECOMMENDATIONS 

Based  upon  the  conclusions  drawn  from  the  statistical  analy- 
sis of  the  data  and  the  issues  raised  by  the  comments  submitted 
by  the  respondents,  the  author  makes  the  following  recommendations 

Because  of  the  relatively  large  number  of  comments  received 
addressing  the  apparent  need  for  a  thorough  reevaluation  of 
9650  billet  requirements,  it  is  recommended  that,  during  the 
next  several  required  annual  reviews  of  billet  educational 
requisites,  the  Operations  Research  discipline  sponsor  (Deputy 
Chief  of  Staff  for  Research,  Development  and  Studies)  elicit 
candid  comments  from  MOS  9650  billet  incumbents  regarding  their 
perceptions  of  proper  billet  utilization.   These  comments  should 
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then  be  incorporated  into  the  normal  billet  review  procedure 
to  enhance  the  picture  obtained  by  the  discipline  sponsor 
concerning  the  true  requirements  for  the  billets. 

Additionally,  and  perhaps  concurrently  with  the  review  of 
educational  requirements,  a  detailed  study  should  be  conducted 
to  determine  if  Marine  Corps  OR  billets  should  be  organized 
on  a  more  centralized  basis.   Where  the  requirement  for  sepa- 
rately assigned  OR  sections  or  individuals  cannot  be  justi- 
fied, it  might  prove  to  be  more  efficient  to  group  them  into 
a  single  section,  perhaps  as  an  Operations  Analysis  Group  under 
the  control  of  the  Deputy  Chief  of  Staff  for  Research,  Develop- 
ment and  Studies.   Under  this  organization,  the  conduct  of 
studies  might  then  be  directed  more  effectively.   In  addition, 
an  overall  plan  for  reviewing  original  negotiations  between 
the  Marine  Corps  and  civilian  contractors,  monitoring  the 
progress  of  the  contracted  studies  and  inspection  of  the  final 
results  could  be  developed  and  employed  by  this  group. 

A  course  of  instruction  regarding  the  management  of  OR 
resources  should  be  offered  to  and  taken  by  all  individuals 
(including  9650' s)  who  are  or  will  be  responsible  for  super- 
vising Marine  Operations  Analysts.   This  course  could  include 
such  subjects  as  the  Potential  and  Limitations  of  Operations 
Analysis,  Management  Techniques,  Contract  Management  and 
Bureaucratic  Realities.   It  could  be  taught  at  the  Marine  Corps 
Command  and  Staff  College,  should  be  at  least  one  to  two  weeks 
long  and  could  be  taken  by  officers  immediately  prior  to 
their  arrival  for  duty  in  a  supervisory  position. 
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A  short  course  in  basic  Operations  Research  could  also 
be  set  up  by  the  Marine  Corps  Extension  School  located  at 
MCDEC,  Quantico,  Virginia.   It  should  embrace,  at  a  minimum, 
the  very  basics  of  OR  to  include  foundational  Probability 
Theory,  simple  Data  Analysis  and  Statistical  Inference,  the 
theory  behind  Linear  Programming,  PERT  and  CPM,  Design  of 
Experiments  and  Technical  Report  Writing.   This  course  could 
be  made  available  to  all  Marines  but  should  be  specifically 
recommended  to  those  officers  who  are  filling  9650  billets 
but  who  do  not  possess  the  9650  secondary  MOS . 

For  those  Marine  officers  attending  the  Naval  Postgraduate 
School  as  a  prerequisite  to  obtaining  a  9650  secondary  specialty, 
the  course  of  instruction  contained  in  Appendix  E  is  submitted 
as  a  means  of  preparing  themselves  for  the  requirements  that 
they  will  find  subsequent  to  graduation.   This  program  is  cer- 
tainly not  all-inclusive  nor  does  it  take  into  consideration 
all  of  the  driving  and  limiting  factors  that  the  administration 
of  the  Naval  Postgraduate  School  must  deal  with  when  developing 
curricula.   However,  this  program  does  contain  most  of  the 
courses  necessary  to  give  the  student  at  least  a  minimal 
knowledge  in  each  of  the  areas  found  by  this  study  to  be 
commonly  used  and  of  importance.   In  addition,  it  is  recommended 
that  the  Operations  Research  Department  Chairman  of  the  Naval 
Postgraduate  School  take  into  consideration  either  the  estab- 
lishment of  a  new  course  dealing  specifically  with  such  topics 
as  Advocacy  Analysis  and  Bureaucratic  Realities  of  Analysis 
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or  at  least  consider  increasing  the  emphasis  placed  upon  such 
subjects  in  existing  NPS  courses. 

Finally,  it  is  recommended  that  the  Special  Education  Pro- 
gram monitor  continue  to  place  emphasis  upon  the  early  iden- 
tification of  9650  billet  vacancies  and  the  pairing  of  these 
openings  with  prospective  analysts  while  they  are  still  attending 
school.   These  pairings  should  be  accomplished  (understanding 
that  last  minute  changes  may  become  necessary)  at  least  six 
months  to  a  year  prior  to  the  graduation  of  the  students. 
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APPENDIX  A 


USMD  OPERATIONS  RESEARCH  SURVEY 


If  you  axe  currently  serving  in  a  MOS  9650  billet,  please  base  your 
answers  to  all  questions  on  present  billet  experiences.   If  you  are  not 
currently  serving  in  a  MOS  965O  billet  but  have  done  so  in  the  past,  please 
base  your  answers  to  all  questions  on  experiences  from  your  last  9650 
billet  assignment .   If  you  have  never  served  in  a  MOS  965O  billet,  please 
state  so  now  and  do  not  answer  any  further  questions.  Otherwise,  please 
continue.  


1.  a.  What  is  the  title  of  your  9650  billet? 


b.  With  what  command  is  this  billet  located? 


2.  What  is  (was)  your  rank  while  serving  in  this  965O  billet? 


3.  Which  of  the  following  general  function  areas  is  this  9650  billet  related 
to?  (Circle  one) 

a.  Manpower  e.  Test  and  Evaluation 

b.  Logistics  f.  Aviation 

c.  Finance  g.  Instruction 

d.  Operations  and  Plans  h.  Other:  What  area?  


k.     What  type  of  Operations  Research  (OR)  work  does  this  9650  billet  require^ 
you  to  do  the  most?  (Circle  one) 

a.  Original  OR  work  d.  Instructor 

b.  Correlate  the  work  of  other        e.  Other  OR  work 
researchers  in  your  section        f .  Do  not  do  any  OR  work 

c.  Review  OR  work  done  outside 
your  section 

5.  Does  this  billet  assignment  require  you  to  do  your  own  computer  programming? 


a.  If  so,  what  computer  language(s)  do  you  use? 


b.  Why  do  you  use  this  particular  language' 


6.  Do  you  interact  with  any  established  computer  programs  (i.e.,  preprogrammed, 
"canned",  manufacturer-provided  software)?  
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7.  How  often  do  you  use  the  following  OR  techniques  and  application  areas? 
(Place  an  X  in  the  appropriate  box) 


a.  General  OR  Techniques 

p 

> 

z 

VI 
M 

o 

CO 

co 

X 

< 
< 

1)  Linear  Programming 

2)  Nonlinear  Programming 

3)  Dynamic  Programming 

b)   Integer  and  Mixed  Programming 

5)  Heuristic  Programming 

6)  3ayesian  Decision  Analysis 

7)  Networks,  Flows  and  Graphs  (PERT/CPM) 

8)  Risk  Analysis 

9)  Cost  Estimation 

10)  Cost  Effectiveness 

11)  Reliability 

12)  Inventory  Theory 

13)  Data  Analysis  and  Statistical  Inference 
(Parametric) 

1*0  Data  Analysis  and  Statistical  Inference 
(Nonparametric) 

15)  Probability  Theory 

b.  OR  Modeling  Techniques 

1)  Queueing  Theory  and  Stochastic  Models 

2)  Gaming  Models 

3)  Combat  Models 

4)  Manpower  Models 

5)  Econometric  Models 

6)  Cost  Models 

7)  Input-Output  Models 

8)  Forecasting. 

c.  Other  Related  Subjects 

1)  Calculus 

2)  Linear  Algebra 

3)  Economics 

k)   Capital  Budgeting 

5)  Policy  Analysis 

6)  Combat  Analysis 

7)  Equipment  Replacement 

8)  Maintenance  and  Repair 

9)  Transportation 

10)  Human  Factors 

11)  Design,  Conduct  and  Evaluation  of  Experiments 

12)  Computer  Programming 
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8.   What  Is  the  relative  Importance  of  having  a  working  knowledge  of  these 
OR  techniques  and  application  areas  in  discharging  the  daily  duties  of 
your  billet?  (On  a  scale  of  1=' Extremely  Important '  to  5= 'Not  Important 
At  All').  Place  an  X  in  the  appropriate  box  for  each  technique  or  subject. 


Ext: 
Imp< 

a.  General  OR  Techniques 

remel 
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Vot 
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ij 

T.lpear  Programming 

Nonlinear  Programming 

3! 

Dynamic  Programming 

* 

Integer  and  Mixed  Programming 

5, 

Heuristic  Programming 

6 

3ayesian  Decision  Analysis 

7, 

Networks,  Flows  and  Graphs  (PERT/GPM) 

8} 

Risk  Analysis 

9 

Cost  Estimation 

10 

Cost  Effectiveness 

11 

Reliability 

12. 

Inventory  Theory_ 

13J 

Data  Analysis  and  Statistical  Inference 
(Parametric} 

to) 

15) 

b.  01 

Data  Analysis  and  Statistical  Inference 
( Nonparame tri c ) 

Probability  Theory 

I   Modeling  Techniques 

1) 

Queueing  Theory  and  Stochastic  Models 

2) 

Gaming  Models 

3, 

Combat  Models 

^( 

Manpower  Models 

5, 

Econometric  Models 

6 

Cost  Models 

7, 

Input-Output  Models 

3; 

Forecasting 

c.  01 

her  Related  Subjects 

ij 

Calculus 

Linear  Algebra 

3, 

Economics 

n. 

Capital  Budgeting 

5, 

Policy  Analysis 

? 

Combat  Analysis 

Equipment  Replacement 

8 

Maintenance  and  Repair 

9, 

Transportation 

10 

Human  Factors 

11 

Design.  Conduct  and  Evaluation  of  Experiments 

12] 

Computer  Programming 
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9.  Other  than  those  listed  above,  what  OR  techniques  and/or  application 
areas  do  you  feel  you  should  have  a  working  knowledge  of  in  order  to 
efficiently  do  your  job?  (Please  list  the  top  five  in  order  of 
importance. ) 


10.  a.  Approximately  what  percentage  of  your  work  time  is  actually  spent 

doing  OR  work  in  comparison  to  your  other  duties?  (Circle  one) 

*0  15% 
5)   100% 

b.  If  your  answer  to  question  10. a.  was  l)  or  2),  in  what  area  is  most 
of  your  work  time  spent?  ___^__^_____________^_______ 

11.  Generally,  what  is  the  acceptance  level  of  those  whom  you  work  for 
towards  OR  work?  (Circle  one) 

a.  OR  work  is  accepted  without  question 

b.  OR  work  is  usually  accepted 

c.  OR  work  is  rarely  accepted 

d.  OR  work  is  never  accepted 

12.  In  your  opinion,  how  could  OR  billets  be  better  utilized?  (Please  use 
the  reverse  side  if  more  space  for  response  is  required. ) 


13.  Do  you  wish  to  receive  a  copy  of  the  results  of  this  survey? 
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APPENDIX    B 
OR   TECHNIQUE    USAGE    AND    IMPORTANCE   FREQUENCIES 


USAGE 


Raw  Score 


Percentage 


03 


CO 


TECHNIQUE 


a.  GENERAL  OR  TECHNIQUES 

1)  Linear  Programming 

2)  Nonlinear  Programming 

3)  Dynamic  Programming 

4)  Integer  and  Mixed  Programming 

5)  Heuristic  Programming 

6)  Bayesian  Decision  Analysis 

7)  Networks  Flows  and  Graphs 

8)  Risk  Analysis 

9)  Cost  Estimation 

10)  Cost  Effectiveness 

11)  Reliability 

12)  Inventory  Theory 

13)  Data  Analysis  and  Statistical 
Inference  (Parametric) 

14)  Data  Analysis  and  Statistical 
Inference  (Nonparametric) 

15)  Probability 

b.  OR  MODELING  TECHNIQUES 

1)  Queueing  Theory  &  Stoch.  Models 

2)  Gaming  Models 

3)  Combat  Models 

4)  Manpower  Models 

5)  Econometric  Models 

6)  Cost  Models 

7)  Input-Output  Models 

8)  Forecasting 

C.   OTHER  RELATED  SUBJECTS 

1)  Calculus 

2)  Linear  Algebra 

3)  Economics 

4)  Capital  Budgeting 

5)  Policy  Analysis 

6)  Combat  Analysis 

7)  Equipment  Replacement 

8)  Maintenance  and  Repair 

9 )  Transportation 

10)  Human  Factors 

11)  Design,  Cond.  &  Eval.  of  Exps. 

12)  Computer  Programming 
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37 

25 

4 

56 

38 

6 

38 

22 

6 

58 

33 

9 

48 

15 

3 

73 

23 

5 

31 

31 

4 

47 

47 

6 

42 

23 

1 

64 

35 

2 

42 

21 

3 

64 

32 

5 

32 

30 

4 

48 

45 

6 

40 

23 

3 

61 

35 

5 

33 

26 

7 

50 

40 

11 

49 

14 

3 

74 

21 

5 

45 

15 

6 

68 

23 

9 

34 

26 

6 

52 

40 

9 

32 

31 

3 

48 

47 

5 

32 

30 

4 

48 

45 

6 

38 

28 

0 

58 

42 

0 

31 

29 

6 

47 

44 

9 

20 

35 

11 

30 

53 

17 

28 

26 

12 

42 

40 

18 
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IMPORTANCE 


TECHNIQUE 


a.  GENERAL  OR  TECHNIQUES 

1)  Linear  Progranming  4   7  18  13  24  6  11  27  20  36 

2)  Nonlinear  Progranming  0   4  10  18  34  0  6  15  27  52 

3)  Dynamic  Progranming  0   1  9  24  32  0  2  14  36  48 

4)  Integer  &  Mixed 

Progranming  0   1  11  15  39  0  2  17  23  59 

5)  Heuristic  Progranming  2   1  8  13  42  3  2  12  20  64 

6)  Bayes.  Dec.  Analy.  1   7  14  14  30  2  11  21  21  45 

7)  Networks,  Flows 

and  Graphs  5  22  21  11  7  8  33  32  17  11 

8)  Risk  Analysis  5  15  16  11  19  8  23  24  17  29 

9)  Cost  Estimation  12  13  17  12  12  18  20  26  18  18 

10)  Cost  Effectiveness  15  16  18  10  7  23  24  27  15  11 

11)  Reliability  10  17  17  12  10  15  26  26  18  15 

12)  Inventory  Theory  2   7  10  22  25  3  11  15  33  38 

13)  Data  Analysis  and 

Stat.  Inf.  (Param.)  22  17  12  7  8  33  26  18  11  12 

14)  Data  Analy.  &  Stat. 

Inf.  (Nonparametric)  21  15  9  11  10  32  23  14  17  15 

15)  Probability  Theory  22  19  9  6  10  33  29  14  9  15 

b.  OR  MODELING  TECHNIQUES 

1)  Queueing  Theory  & 

Stochastic  Models  2  12  15  18  19  3  18  23  27  29 

2)  Gaming  Models  5  16  9  15  21  8  24  14  "23  32 

3)  Combat  Models  9  12  6  12  27  14  18  9  18  41 

4)  Manpower  Models  12   5  13  18  18  18  8  20  27  27 

5)  Econometric  Models  2  11  12  19  22  3  17  18  29  33 

6)  Cost  Models  8  19  13  14  12  12  29  20  21  18 

7)  1-0  Models  5  11  7  16  27  8  17  11  24  41 

8)  Forecasting  7  13  11  12  23  11  20  17  18  35 

c.  OTHER  RELATED  SUBJECTS 

1)  Calculus  8  17  13  11  17  12  11  20  17  26 

2)  Linear  Algebra  4  10  24  9  19  6  15  36  14  29 

3)  Economics  11  15  13  14  13  17  23  20  21  20 

4)  Capital  Budgeting  2   9  14  15  26  3  14  21  23  40 

5)  Policy  Analysis  6  11  14  16  19  9  17  21  24  29 

6)  Combat  Analysis  9  11  13  8  25  14  17  20  12  38 

7)  Equipment  Repl.  2  16  11  16  21  3  24  17  24  32 

8)  Maintenance  &  Repair  4  12  15  16  19  6  18  23  24  29 

9)  Transportation  1  13  11  21  20  2  20  17  32  30 

10)  Human  Factors  11  13  13  11  18  17  20  20  17  27 

11)  Design,  Cond.  & 

Eval.  of  Exps.  23  13  6  11  13  35  20  9  17  20 

12)  Computer  Progr.  18  16  12  9  11  27  24  18  14  17 
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APPENDIX  C 
USAGE  FREQUENCIES  BY  SERVICE  RANK.  FREQUENCY--CAPT/MAJ  =  52 


Paw  Score 


Percentage 


TECHNIQUE 


Eh 


03 


H 

Eh 


a.  GENERAL  OR  TECHNIQUES 

1)  Linear  Progr  arming 

2)  Nonlinear  Programming 

3)  Dynamic  Programming 

4)  Integ.  &  Mixed  Programming 

5)  Heuristic  Programming 

6)  Bayesian  Decision  Analysis 

7)  Networks,  Flows  and  Graphs 

8)  Risk  Analysis 

9)  Cost  Estimation 

10)  Cost  Effectiveness 

11)  Reliability 

12)  Inventory  Theory 

13)  Data  Analy.  &  Statistical 
Inference  (Parametric) 

14)  Data  Analy.  &  Statistical 
Inference  (Nonparametric) 

15)  Probability  Theory 

b.  OR  MODELING  TECHNIQUES 

1)  Queueing  Theory  &  Stoch.  Models 

2)  Gaming  Models 

3)  Combat  Models 

4)  Manpower  Models 

5)  Econometric  Models 

6)  Cost  Models 

7)  1-0  Models 

8)  Forecasting 

C.   OTHER  RELATED  SUBJECTS 

1)  Calculus 

2)  Linear  Algebra 

3)  Economics 

4)  Capital  Budgeting 

5)  Policy  Analysis 

6)  Combat  Analysis 

7)  Equipment  Replacement 

8)  Maintenance  and  Repair 

9)  Transportation 

10)  Human  Factors 

11)  Design,  Cond.  &  Eval.  of  Exps. 

12)  Computer  Programming 


34 
45 
46 
49 
46 
38 
10 
31 
14 
12 
19 
41 


19 
13 


32 
34 
31 
30 
36 
23 
34 
30 


25 
30 
23 
42 
38 
28 
25 
25 
29 
25 
16 
19 


18 

7 

5 

3 

5 

14 

41 

20 

34 

33 

26 

10 


22 
26 


19 
16 
18 
17 
13 
25 
18 
19 


23 
19 
25 
9 
12 
20 
24 
23 
23 
22 
27 
22 


0 
0 
1 
0 

1 

0 

1 
1 

4 
7 

7 
1 


11   28   13 


11 
13 


4 
3 
4 
1 
2 
4 
3 
4 
0 
5 
9 
11 


65 
87 
88 
94 
88 
73 
19 
60 
27 
23 
37 
79 


37 
25 


62 
65 
60 
58 
69 
44 
65 
58 


48 
58 
44 
81 
73 
54 
48 
48 
56 
48 
31 
37 


35 
13 
10 
6 
10 
27 
79 
38 
65 
63 
50 
19 


21   54 


42 
50 


37 
31 
35 
33 
25 
48 
35 
37 


44 
37 
48 
17 
23 
38 
46 
44 
44 
42 
52 
42 


0 
0 
2 
0 
2 
0 
2 
2 
8 
13 
13 
2 

25 

21 
25 


2 
4 
6 
10 
6 
8 
0 
6 


8 

6 

8 

2 

4 

8 

6 

8 

0 

10 

17 

21 
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Frequency — LT.  Col. /Col.  =  14 

Raw  Score 


Percentage 


03 


03 


TECHNIQUE 


O 

03 


a.  GENERAL  OR  TECHNIQUES 

1)  Linear  Programming 

2)  Nonlinear  Programming 

3)  Dynamic  Programming 

4)  In  teg.  &  Mixed  Programming 

5)  Heuristic  Programming 

6)  Bayesian  Decision  Analysis 

7)  Networks,  Flows  and  Graphs 

8)  Risk  Analysis 

9)  Cost  Estimation 

10)  Cost  Effectiveness 

11)  Reliability 

12)  Inventory  Theory 

13)  Data  Analy.  &  Statistical 
Inference  (Parametric) 

14)  Data  Analy.  St   Statistical 
Inference  ( Nonpar ame trie) 

15)  Probability  Theory 

b.  OR  MODELING  TECHNIQUES 

1)  Queueing  Theory  &  Stoch.  Models 

2)  Gaming  Models 

3)  Combat  Models 

4)  Manpower  Models 

5)  Econometric  Models 

6)  Cost  Models 

7)  1-0  Models 

8)  Forecasting 

c.  OTHER  RELATED  SUBJECTS 

1)  Calculus 

2)  Linear  Algebra 

3)  Economics 

4)  Capital  Budgeting 

5)  Policy  Analysis 

6)  Combat  Analysis 

7)  Equipment  Replacement 

8)  Maintenance  and  Repair 

9 )  Transportation 

10)  Human  Factors 

11)  Design,  Conduct  &  Eval.  of  Exps. 

12)  Computer  Prcgramming 


9 

11 

11 

12 

13 

10 

4 

6 

4 

1 

6 

11 


8 
7 
6 
8 

12 
8 
8 

12 


7 
10 
10 
7 
7 
6 
7 
7 
9 
6 
4 
9 


5 
3 
3 
2 
1 
4 
8 
6 
9 
10 
7 
2 

10 

9 

11 


0 
0 
0 
0 
0 
0 
2 
2 
1 
3 
1 
1 


64 
79 
79 
86 
93 
71 
29 
43 
29 
7 
43 
79 

14 

21 

14 


57 
50 
43 
57 
86 
57 
57 
86 


50 
71 
71 
50 
50 
43 
50 
50 
64 
43 
29 
64 


36 
21 
21 
14 
7 
29 
57 
43 
64 
71 
50 
14 


64 
79 


43 
43 
50 
36 
14 
43 
36 
14 


50 
29 
7 
36 
21 
43 
50 
50 
36 
50 
57 
29 


0 

0 

0 

0 

0 

0 

14 

14 

7 

21 

7 

7 


71   14 


14 
7 


0 

0 

21 

14 

29 

14 

0 

0 

0 

7 

14 

7 
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APPENDIX  D 

IMPORTANCE  FREQUENCIES  BY  SERVICE  RANK.  Importance-- Capt/Maj  =  52 

Raw  Score  Percentage 


TECHNIQUE 


a.  GENERAL  OR  TECHNIQUES 

1)  Linear  Programming 

2)  Nonlinear  Program. 

3)  Dynamic  Programming 

4)  Integ.  &  Mixed  Prog. 

5)  Heuristic  Programming 

6)  Bayesian  Dec.  Analy. 

7)  Networks,  Flows 
and  Graphs 

8)  Risk  Analysis 

9)  Cost  Estimation 

10)  Cost  Effectiveness 

11)  Reliability 

12)  Inventory  Theory 

13)  Data  Analy.  &  Stat. 
Inference  (Par am. ) 

14)  Data  Analy.  &  Stat. 
Inference  (Nonparam.)  17 

15)  Probability  Theory 

b.  OR  MODELING  TECHNIQUES 

1)  Queueing  Theory  & 
Stochastic  Models 

2)  Gaming  Models 

3)  Combat  Models 

4)  Manpower  Models 

5)  Econometric  Models 

6)  Cost  Models 

7)  1-0  Models 

8)  Forecasting 

C.   OTHER  RELATED  SUBJECTS 

1)  Calculus 

2)  Linear  Algebra 

3)  Economics 

4)  Capital  Budgeting 

5)  Policy  Analysis 

6)  Combat  Analysis 

7)  Equipment  Repl. 

8)  Maintenace  &  Repair 

9)  Transportation        1   11    9   17   14    2   21   17   33 

10)  Human  Factors        9   10   11    8   14   17   19   21   15   27 

11)  Des.,  Cond.  &Evl.  Exp. 18   11    5    7   11   35   21   10   13   21 

12)  Computer  Programming  16   12    8    7    9   31   23   15   13   17 
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2 

5 

15 

10 

20 

4 

10 

29 

19 

38 

0 

3 

6 

14 

29 

0 

6 

12 

27 

56 

0 

1 

5 

19 

27 

0 

2 

10 

37 

52 

0 

1 

7 

11 

33 

0 

2 

13 

21 

63 
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1 

5 

9 

35 
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17 
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9 
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0 

12 
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12 
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16 
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14 

7 

9 

19 

6 

27 

13 

17 

37 

7 

11 

3 

7 

24 

13 

21 

6 

13 

46 

10 

5 

9 

12 

16 

19 

10 

17 

23 

31 

2 

10 

10 

12 

18 
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19 

19 

23 

35 

7 

15 

10 

9 
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13 
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19 

17 

21 

5 

8 

6 

9 

24 

10 

15 

12 

17 

46 

6 

12 

8 

8 

18 

12 

23 

15 

15 

35 
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19 
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20 

0 

17 
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38 

2 

11 

12 

13 

14 

4 

21 

23 

25 

27 

6 

9 

11 

4 

22 

12 

17 

21 

8 

42 

2 

14 

8 

12 

16 

4 

27 

15 

23 

31 

4 

11 

12 

10 

15 

8 

21 

23 

19 

29 

Importance — LT  Col/Col  =  14 
Raw  Score 


Percentage 


3S 

4-> 

>14J 

4J 

u 

S  4J 

53  u 

s 

18. 

-p  a 

u   3 

-u  a 

Cm  H 

2  H 

3  H 

2  M 

TECHNIQUE 

1 

2 

3 

4 

5 

1 

2 

3 

4 

5 

a.  GENERAL  OR  TECHNIQUES 

1) 

Linear  Programming 

2 

2 

3 

3 

4 

14 

14 

21 

21 

29 

2) 

Nonlinear  Programming 

0 

1 

4 

4 

5 

0 

7 

29 

29 

36 

3) 

Dynamic  Programming 

0 

0 

4 

5 

5 

0 

0 

29 

36 

36 

4) 

Integ.  &  Mixed  Prog. 

0 

0 

4 

4 

6 

0 

0 

29 

29 

43 

5) 

Heuristic  Programming 

0 

0 

3 

4 

7 

0 

0 

21 

29 

50 

6) 

Bayesian  Dec.  Analy. 

1 

1 

3 

5 

4 

7 

7 

21 

36 

29 

7) 

Networks,  Flows 

and  Graphs 

1 

3 

5 

3 

2 

7 

21 

36 

21 

14 

8) 

Risk  Analysis 

1 

3 

3 

2 

5 

7 

21 

21 

14 

36 

9) 

Cost  Estimation 

1 

2 

4 

4 

3 

7 

14 

29 

29 

21 

10) 

Cost  Effectiveness 

2 

4 

5 

2 

1 

14 

29 

36 

14 

7 

11) 

Reliability 

1 

5 

2 

5 

1 

7 

36 

14 

36 

7 

12) 

Inventory  Theory 

1 

0 

2 

6 

5 

7 

9 

14 

43 

36 

13) 

Data  Analy.  &  Stat. 

Inf.  (Parametric) 

3 

3 

4 

2 

2 

21 

21 

29 

14 

14 

14) 

Data  Analy.  &  Stat. 

Inf.  (Ndnparametric) 

4 

3 

3 

2 

2 

29 

21 

21 

14 

14 

15) 

Probability  Theory 

2 

5 

4 

2 

1 

14 

36 

29 

14 

7 

b.   OR  MODELING  TECHNIQUES 

1) 

Queueing  Theory  & 

- 

Stochastic  Models 

1 

2 

3 

5 

3 

7 

14 

21 

36 

21 

2) 

Gaming  Models 

2 

2 

2 

6 

2 

14 

14 

14 

43 

14 

3) 

Combat  Models 

2 

1 

3 

5 

3 

14 

7 

21 

36 

21 

4) 

Manpower  Models 

2 

0 

4 

6 

2 

14 

0 

29 

43 

14 

5) 

Econometric  Models 

0 

1 

2 

7 

4 

0 

7 

14 

50 

29 

6) 

Cost  Models 

1 

4 

3 

5 

1 

7 

29 

21 

36 

7 

7) 

1-0  Models 

0 

3 

1 

7 

3 

0 

21 

7 

50 

21 

8) 

Forecasting 

1 

1 

3 

4 

5 

7 

7 

21 

29 

36 

C.   OIHER  RELATED  SUBJECTS 

1) 

Calculus 

1 

4 

4 

2 

3 

7 

29 

29 

14 

21 

2) 

Linear  Algebra 

0 

3 

5 

3 

3 

0 

21 

36 

21 

21 

3) 

Economics 

3 

1 

3 

4 

3 

21 

7 

21 

29 

21 

4) 

Capital  Budgeting 

2 

0 

3 

3 

6 

14 

0 

21 

21 

43 

5) 

Policy  Analysis 

4 

0 

2 

3 

5 

29 

0 

14 

21 

36 

6) 

Combat  Analysis 

3 

2 

2 

4 

3 

21 

14 

14 

29 

21 

7) 

Equipment  Repl. 

0 

2 

3 

4 

5 

0 

14 

21 

29 

36 

8) 

Maintenance  &  Repair 

0 

1 

3 

6 

4 

0 

7 

21 

43 

29 

9) 

Transportation 

0 

2 

2 

4 

6 

0 

14 

14 

29 

43 

10) 

Human  Factors 

2 

3 

2 

3 

4 

14 

21 

14 

21 

29 

11) 

Des.,  Cond.  &Evl.  Exp. 

5 

2 

1 

4 

2 

36 

14 

7 

29 

14 

12) 

Computer  Programming 

2 

4 

4 

2 

2 

14 

29 

29 

14 

14 
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APPENDIX  E 

EXAMPLE  OF  AN  OPERATIONS  ANALYSIS 
CURRICULUM  FOR  U.S.M.C.  STUDENTS 
ATTENDING  THE  NAVAL  POSTGRADUATE 
SCHOOL 


First  Quarter 

Introductory  Computing  for  Operations  Analysis 

Topics  in  Calculus 

Probability 

History  &  Nature  of  Operations  Analysis 

Linear  Algebra 

Seminar  for  OA  students 

Second  Quarter 

Linear  Programming 

Selected  Topics  from  Advanced  Calculus 
Probability  and  Statistics 
Introduction  to  Mathematical  Economics 
Seminar  for  OA  students 

Third  Quarter 

Statistics 
Stochastic  Models  I 
Computational  Methods  for  OR 
Networks,  Flows  and  Graphs 
Seminar  for  OA  students 

Fourth  Quarter 

Utility  Theory  &  Resource  Allocation 
Human  Factors  in  System  Design  I 
Analysis  of  Operational  Data 
Systems  Simulation 
Seminar  for  OA  students 

« 

Fifth  Quarter  (includes  6  week  Experience  Tour). 

Systems  Analysis  I 
Combat  Models  &  Wargaming 
Seminar  for  OA  students 

Sixth  Quarter 

Non-Linear  and  Dynamic  Programming 
Stochastic  Models  II 
Design  of  Experiments 
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Elective 

Seminar  for  OA  students 

Seventh  Quarter 

Thesis  Research 

Reliability  and  Weapon  System  Effectiveness 

Evaluation  of  Human  Factors  Data 

Elective 

Seminar  for  OA  students 

Eighth  Quarter 

Thesis  Research 

Cost  Estimation 

Elective 

Elective 

Seminar  for  OA  students 
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